-- Logs begin at Thu 2022-07-14 14:50:02 UTC, end at Thu 2022-07-14 16:33:24 UTC. -- Jul 14 15:21:28.113810 dv-cinder-826510-8-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 14 15:21:28.123694 dv-cinder-826510-8-iscsi devstack@keystone.service[80227]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 14 15:21:28.126275 dv-cinder-826510-8-iscsi devstack@keystone.service[80227]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Jul 14 15:21:28 2022] *** Jul 14 15:21:28.126275 dv-cinder-826510-8-iscsi devstack@keystone.service[80227]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 14 15:21:28.126275 dv-cinder-826510-8-iscsi devstack@keystone.service[80227]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 14 15:21:28.126275 dv-cinder-826510-8-iscsi devstack@keystone.service[80227]: nodename: dv-cinder-826510-8-iscsi Jul 14 15:21:28.126275 dv-cinder-826510-8-iscsi devstack@keystone.service[80227]: machine: x86_64 Jul 14 15:21:28.126275 dv-cinder-826510-8-iscsi devstack@keystone.service[80227]: clock source: unix Jul 14 15:21:28.126275 dv-cinder-826510-8-iscsi devstack@keystone.service[80227]: pcre jit disabled Jul 14 15:21:28.126275 dv-cinder-826510-8-iscsi devstack@keystone.service[80227]: detected number of CPU cores: 4 Jul 14 15:21:28.126275 dv-cinder-826510-8-iscsi devstack@keystone.service[80227]: current working directory: / Jul 14 15:21:28.126275 dv-cinder-826510-8-iscsi devstack@keystone.service[80227]: detected binary path: /usr/bin/uwsgi-core Jul 14 15:21:28.126275 dv-cinder-826510-8-iscsi devstack@keystone.service[80227]: your processes number limit is 47680 Jul 14 15:21:28.126275 dv-cinder-826510-8-iscsi devstack@keystone.service[80227]: your memory page size is 4096 bytes Jul 14 15:21:28.127150 dv-cinder-826510-8-iscsi devstack@keystone.service[80227]: detected max file descriptor number: 2048 Jul 14 15:21:28.127150 dv-cinder-826510-8-iscsi devstack@keystone.service[80227]: lock engine: pthread robust mutexes Jul 14 15:21:28.127150 dv-cinder-826510-8-iscsi devstack@keystone.service[80227]: thunder lock: enabled Jul 14 15:21:28.127150 dv-cinder-826510-8-iscsi devstack@keystone.service[80227]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 14 15:21:28.127150 dv-cinder-826510-8-iscsi devstack@keystone.service[80227]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 14 15:21:28.172072 dv-cinder-826510-8-iscsi devstack@keystone.service[80227]: Python main interpreter initialized at 0x557938cd29c0 Jul 14 15:21:28.172072 dv-cinder-826510-8-iscsi devstack@keystone.service[80227]: python threads support enabled Jul 14 15:21:28.172072 dv-cinder-826510-8-iscsi devstack@keystone.service[80227]: your server socket listen backlog is limited to 100 connections Jul 14 15:21:28.172072 dv-cinder-826510-8-iscsi devstack@keystone.service[80227]: your mercy for graceful operations on workers is 90 seconds Jul 14 15:21:28.172536 dv-cinder-826510-8-iscsi devstack@keystone.service[80227]: mapped 403077 bytes (393 KB) for 2 cores Jul 14 15:21:28.172642 dv-cinder-826510-8-iscsi devstack@keystone.service[80227]: *** Operational MODE: preforking *** Jul 14 15:21:28.172940 dv-cinder-826510-8-iscsi devstack@keystone.service[80227]: *** uWSGI is running in multiple interpreter mode *** Jul 14 15:21:28.172940 dv-cinder-826510-8-iscsi devstack@keystone.service[80227]: spawned uWSGI master process (pid: 80227) Jul 14 15:21:28.173159 dv-cinder-826510-8-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 14 15:21:28.174296 dv-cinder-826510-8-iscsi devstack@keystone.service[80227]: spawned uWSGI worker 1 (pid: 80236, cores: 1) Jul 14 15:21:28.174296 dv-cinder-826510-8-iscsi devstack@keystone.service[80227]: spawned uWSGI worker 2 (pid: 80237, cores: 1) Jul 14 15:21:28.174296 dv-cinder-826510-8-iscsi devstack@keystone.service[80227]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 14 15:21:33.733500 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 14 15:21:33.733500 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: from cryptography.utils import int_from_bytes Jul 14 15:21:33.734303 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 14 15:21:33.734303 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: from cryptography.utils import int_from_bytes Jul 14 15:21:33.743493 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 14 15:21:33.743493 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: from cryptography.utils import int_from_bytes Jul 14 15:21:33.744027 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 14 15:21:33.744027 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: from cryptography.utils import int_from_bytes Jul 14 15:21:33.785634 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 14 15:21:33.785818 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 14 15:21:33.786009 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] command line args: [] {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 14 15:21:33.786283 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 14 15:21:33.786465 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ================================================================================ {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 14 15:21:33.786725 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] admin_token = **** {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:21:33.786974 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] config_dir = [] {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:21:33.787167 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:21:33.787353 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] config_source = [] {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:21:33.787540 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] debug = True {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:21:33.787771 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:21:33.788294 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:21:33.788386 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:21:33.788573 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:21:33.788824 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:21:33.789057 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] list_limit = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:21:33.789243 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] log_config_append = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:21:33.789425 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:21:33.789643 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] log_dir = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:21:33.789768 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] log_file = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:21:33.790010 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:21:33.790260 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:21:33.790508 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:21:33.790755 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:21:33.790937 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:21:33.791133 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:21:33.791364 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:21:33.791637 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:21:33.791788 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 14 15:21:33.791880 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 14 15:21:33.791956 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:21:33.792026 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] command line args: [] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 14 15:21:33.792086 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:21:33.792288 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 14 15:21:33.792380 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:21:33.792494 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ================================================================================ {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 14 15:21:33.792618 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:21:33.792732 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] admin_token = **** {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:21:33.792824 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:21:33.792980 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] config_dir = [] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:21:33.793069 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:21:33.793173 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:21:33.793282 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:21:33.793360 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] config_source = [] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:21:33.793526 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] debug = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:21:33.793740 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:21:33.793843 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.99/identity {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:21:33.793945 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] publish_errors = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:21:33.794034 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:21:33.794148 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:21:33.794274 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:21:33.794402 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:21:33.794478 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:21:33.794583 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:21:33.794665 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:21:33.794832 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:21:33.794909 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] list_limit = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:21:33.795054 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] log_config_append = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:21:33.795134 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:21:33.795236 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:21:33.795318 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] standard_threads = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:21:33.795419 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] log_dir = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:21:33.795516 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:21:33.795616 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] log_file = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:21:33.795791 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:21:33.795876 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:21:33.795972 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:21:33.796092 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:21:33.796169 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] use_journal = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:21:33.796317 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] use_json = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:21:33.796391 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:21:33.796495 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] use_stderr = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:21:33.796577 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:21:33.796679 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] use_syslog = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:21:33.796757 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:21:33.796895 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:21:33.796971 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:21:33.797144 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:21:33.797230 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.797422 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.797576 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:21:33.798060 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:21:33.798142 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.798142 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.798283 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:21:33.798355 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.798424 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:21:33.798492 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.798587 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:21:33.798719 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.798837 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:21:33.798962 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.799087 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:21:33.799217 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.799368 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:21:33.799454 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.799739 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.799898 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.99/identity {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:21:33.799989 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.800070 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] publish_errors = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:21:33.800219 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.800298 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:21:33.800459 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.800537 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:21:33.800697 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:21:33.800775 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.800947 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:21:33.801024 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.801183 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.801266 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:21:33.801417 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] standard_threads = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:21:33.801500 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.801653 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:21:33.801731 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.801901 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:21:33.801985 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.802085 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:21:33.802163 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.802266 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] use_journal = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:21:33.802392 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.802475 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] use_json = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:21:33.802615 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] use_stderr = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:21:33.802703 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.802796 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] use_syslog = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:21:33.802901 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.803030 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:21:33.803152 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.803289 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.803373 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.803533 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.803639 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.803829 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.803906 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.804075 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.804235 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.804311 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.804428 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.804558 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.804642 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.804791 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.804875 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.805024 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.805100 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.805273 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.805350 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.805522 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.805606 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.805760 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.805835 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.806002 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.806084 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.806236 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.806320 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.806476 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.806553 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.806715 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.806798 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.806948 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.807037 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.807181 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.807258 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.807414 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.807492 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.807673 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.807780 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.807917 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.807996 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.808168 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.808245 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.808403 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.808527 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.808642 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.808720 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.808922 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.809015 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.809095 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.809167 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.809338 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.809414 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.809588 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.809665 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.809809 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.809882 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.810049 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.810132 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.810287 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.810364 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.810525 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.810603 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.810763 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.810839 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.810996 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.811072 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.811227 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.811303 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.811477 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.811553 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.811747 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.811854 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.811984 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.812064 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.812224 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.812318 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.812464 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.812529 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.812704 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.812778 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.813449 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.813561 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.814201 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.814313 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.814398 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.814476 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.814596 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.814672 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.814795 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.814867 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.814947 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.815023 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.815140 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.815216 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.815331 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.815412 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.815493 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.815575 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.815686 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.815819 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.815897 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.816135 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.816217 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.816371 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.816489 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.816613 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.816692 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.816855 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.816942 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.817095 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.817175 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.817335 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.817415 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.817574 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.817653 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.817808 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.817888 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.818054 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.818134 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.818291 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.818368 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.818535 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.818615 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.818773 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.818866 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.819011 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.819089 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.819260 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.819340 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.819501 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.819580 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.819764 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.819853 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.820002 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.820081 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.820240 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.820316 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.820476 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.820555 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.820712 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.820791 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.820950 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.821027 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.821186 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.821270 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.821431 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.821511 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.821668 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.821744 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.821917 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.821994 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.822158 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.822237 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.822397 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.822476 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.822642 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.822720 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.822884 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.822975 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.823124 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] auth.external = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.823202 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.823360 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.823438 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.823622 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.823706 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.823864 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.823945 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.824127 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] auth.password = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.824237 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.824362 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] auth.token = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.824487 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.824606 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.824736 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.824844 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.824979 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.825093 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.825227 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.825313 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.825469 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.825568 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.825710 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.825817 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.825951 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.826051 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.826187 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.826288 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] credential.caching = True {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.826421 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.826538 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.826646 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.826791 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.826879 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.827045 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.827125 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.827287 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.827371 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.827536 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.827675 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.827803 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.827883 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.828064 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.828141 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.828321 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.828408 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.828583 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.829507 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.829507 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.829959 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.830046 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] auth.external = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.830046 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.830240 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.830313 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.830385 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.830456 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.830550 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.830638 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] auth.password = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.830804 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] auth.token = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.830888 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.831049 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.831128 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.831288 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.831375 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] federation.caching = True {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.831531 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.831630 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.831789 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.831867 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.832042 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.832134 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.832278 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.832357 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.832519 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.832605 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.832761 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.832853 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] credential.caching = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.833013 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.833091 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.833254 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.833335 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.833512 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.833599 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.833750 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.833834 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.834003 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.834082 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.834243 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.834328 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] identity.caching = True {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.834477 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.834558 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.834732 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.834811 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.834973 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.835060 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.835213 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.835293 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.835463 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.835542 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.835730 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.835823 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.835972 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.836057 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.836225 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.836304 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.836480 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.836568 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.836717 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.836796 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] federation.caching = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.836950 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.837029 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.837185 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.837346 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.837422 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.837494 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.837662 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.837740 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.837913 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.838001 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.838155 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.838234 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.838407 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.838485 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.838644 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.838723 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.838903 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.838982 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.839141 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.839218 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.839376 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.839456 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.839635 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.839744 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] identity.caching = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.839879 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.839970 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.840115 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.840204 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.840354 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.840432 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.840594 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.840673 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.840843 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.840922 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.841076 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.841153 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.841319 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.841398 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.841566 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.841645 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.841804 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.841882 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.842037 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.842113 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.842276 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.842355 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.842513 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.842589 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.842772 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.842940 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.843074 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.843212 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.843340 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.843468 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.843608 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.843732 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.843869 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.843997 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.844123 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.844271 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.844797 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.844920 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.845051 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.845179 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.845179 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.845429 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.845558 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.845685 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.845820 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.845947 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.846074 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.846258 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.846382 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.846510 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.846647 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.846775 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.846907 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.847032 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.847158 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.847293 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.847429 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.847555 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.user = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.847728 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.847854 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.847980 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.848106 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.848242 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.848369 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.848557 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.848679 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.848803 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.848930 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.849075 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.849202 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.849327 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.849452 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.849577 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.849702 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.849837 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.849964 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.850090 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.850221 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.850347 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.850480 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.850607 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.850733 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.850733 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.850984 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.851111 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.851264 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.851264 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.851500 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.851638 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.851761 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.851889 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.852024 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.852024 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.852261 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.852386 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.852516 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.852645 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.852780 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.852907 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.853031 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.853157 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.853281 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.user = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.853407 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.853539 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.853665 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.853790 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.853915 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.854041 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.854167 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.854309 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.854435 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.854523 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.854682 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.854807 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.854933 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.855068 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.855195 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.855321 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.855445 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.855571 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.855711 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.855847 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.855974 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.856111 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.856244 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.856370 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.856505 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.856638 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.856764 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.856889 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.857020 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] resource.caching = True {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.857147 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.857273 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.857406 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.857537 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.857663 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.857788 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.857923 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.858048 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.858182 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.858310 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.858435 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.858559 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.858684 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.858684 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.858928 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.859054 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.859179 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.859179 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.859416 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.859551 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] role.caching = True {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.859733 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.859733 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.860848 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] role.driver = sql {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.860848 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.861093 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.861221 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.861358 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.861482 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.861609 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.861609 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.861847 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.861973 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.862107 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.862243 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.862371 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.862496 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.862623 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.862749 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.862947 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.863075 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.863075 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.863312 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.863439 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] resource.caching = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.863564 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.863717 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.863841 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.863973 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.864096 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.864218 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.864340 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.864471 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.864595 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.864717 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.864717 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.864945 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.865074 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.865197 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.865320 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.865441 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.865565 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.865696 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.865827 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] role.caching = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.865950 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.866070 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] role.driver = sql {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.866194 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.866319 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.866690 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.866823 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.866946 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.867076 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.867198 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.867320 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.867441 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.867582 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.867738 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.867865 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.867991 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.868116 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.868242 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.868378 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.868505 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.868632 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.868757 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.868883 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.869008 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.869143 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.869270 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.869406 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.869532 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.869659 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.869785 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.869919 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.870046 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] token.caching = True {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.870046 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.870283 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.870408 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.870533 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.870671 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.870799 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.870926 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.871062 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.871188 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.871313 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.871446 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.871573 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.871741 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.871864 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.871987 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.872119 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.872242 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.872374 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.872497 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.872618 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.872751 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.872882 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.873006 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.873127 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.873249 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.873369 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.873491 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.873620 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.873744 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.873866 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.873988 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.874109 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.874230 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.874360 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.874496 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] database.connection = **** {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.874626 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.874626 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.874866 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.874989 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] token.caching = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.874989 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.875217 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.875347 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.876136 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.876634 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.876762 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.876885 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.877014 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.877156 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.877294 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.877422 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.877549 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.877675 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.877802 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.877936 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.878065 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.878192 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.878319 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.878444 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.878570 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.878704 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.878831 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.878971 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.879098 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.879225 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.879351 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.879485 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.879631 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.879756 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.879878 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.880000 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.880122 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.880260 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.880384 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.880384 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80236) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 14 15:21:33.880616 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] database.connection = **** {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.880616 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.880616 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.880616 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.880616 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.880616 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.880616 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.880616 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.880616 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.881629 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.881629 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.881629 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.881629 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.881629 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.881629 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.881629 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.881629 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.881629 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.882637 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:21:33.882637 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 14 15:21:33.907696 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80236) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:21:33.907930 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80236) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:21:33.908159 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80236) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:21:33.910638 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:21:33.910878 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:21:33.911111 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:21:33.916750 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80236) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:21:33.919632 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:21:33.946378 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80236) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:21:33.946611 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80236) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:21:33.946939 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:21:33.947179 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:21:34.095014 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:21:34.100261 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80236) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:21:34.145300 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:21:34.145537 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:21:34.146597 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80236) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:21:34.146837 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80236) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:21:34.211888 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:21:34.212115 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:21:34.212342 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:21:34.212500 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80236) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:21:34.212608 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:21:34.212737 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80236) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:21:34.212819 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:21:34.212963 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80236) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:21:34.213044 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:21:34.213184 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80236) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:21:34.213403 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80236) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:21:34.213618 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80236) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:21:34.238965 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:21:34.239201 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:21:34.239390 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80236) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:21:34.239489 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:21:34.239570 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80236) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:21:34.239715 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:21:34.239803 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80236) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:21:34.239936 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:21:34.240023 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80236) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:21:34.240153 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:21:34.240236 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80236) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:21:34.240371 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:21:34.240454 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80236) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:21:34.240618 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80236) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:21:34.252492 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:21:34.252808 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80236) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:21:34.265447 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:21:34.265681 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:21:34.265792 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80236) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:21:34.265951 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:21:34.266035 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80236) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:21:34.266205 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:21:34.266350 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80236) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:21:34.266482 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:21:34.266565 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80236) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:21:34.267255 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:21:34.267352 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80236) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:21:34.267428 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:21:34.267502 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80236) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:21:34.267584 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80236) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:21:34.285038 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.285252 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 14 15:21:34.285341 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.285491 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 14 15:21:34.285642 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.285803 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.286020 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 14 15:21:34.286245 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 14 15:21:34.286327 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.286519 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.286609 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.286781 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.286862 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.287038 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.287120 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.287295 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.298058 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.298354 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.298439 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.298632 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.298714 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.298899 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.298982 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.299179 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.307250 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80237) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 15:21:34.307548 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80236) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 15:21:34.310576 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80237) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 15:21:34.310856 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.311113 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80236) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 15:21:34.311196 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.311398 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.311492 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.311706 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.311788 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.311972 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.312052 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.312243 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.312326 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.312499 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.312578 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.312768 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.312847 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.313039 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.313622 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.313822 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.313941 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.314078 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.314439 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.341091 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.341540 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.342718 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80237) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 15:21:34.342999 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.343155 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80236) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 15:21:34.343435 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.348533 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80237) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 15:21:34.348779 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80236) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 15:21:34.348915 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.349059 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.349152 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.349338 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.356531 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80237) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 15:21:34.356771 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80236) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 15:21:34.356856 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.357053 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.361341 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80237) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 15:21:34.361622 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.361705 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80236) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 15:21:34.361909 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.362002 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.362191 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.362273 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.362471 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.362551 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.362746 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.362847 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.363015 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.363096 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.363281 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.363361 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.363549 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.382799 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.383079 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.383315 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.383591 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.387643 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80237) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 15:21:34.388274 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80236) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 15:21:34.392028 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.392589 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.392679 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.392770 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.397657 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80237) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 15:21:34.397947 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80236) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 15:21:34.401345 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80237) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 15:21:34.401614 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80236) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 15:21:34.405046 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.405314 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.405397 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.405615 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.405697 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.405907 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.406003 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.406186 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.406268 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.406465 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.406555 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.406745 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.427948 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80236) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 15:21:34.428236 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.428501 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.428761 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.429518 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80237) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 15:21:34.429816 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.430087 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.430353 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.436720 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.439062 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.439182 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.439440 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.441004 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.441285 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.442687 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80236) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 15:21:34.442964 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.443244 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.443524 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.443827 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.444547 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80237) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 15:21:34.444831 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.445111 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.445392 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.445678 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.457981 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80236) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 15:21:34.458260 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.458540 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.458820 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.459110 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.459397 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.459588 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80237) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 15:21:34.459690 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.459891 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.460175 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.460457 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.460758 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.461051 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.461332 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.479643 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80236) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 15:21:34.480903 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80237) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 15:21:34.482936 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80236) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 15:21:34.484228 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80237) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 15:21:34.486392 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.487582 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.488025 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.489219 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.489792 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80236) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 15:21:34.490069 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.490354 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.491001 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80237) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 15:21:34.491283 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.491576 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.497902 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.499182 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.499535 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80236) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 15:21:34.501399 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80237) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 15:21:34.503998 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.504227 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.504510 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.504792 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.506322 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.506614 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.506897 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.507174 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.514033 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80236) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 15:21:34.514320 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.514601 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.517115 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80237) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 15:21:34.517115 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.517633 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.523201 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80236) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 15:21:34.523478 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.523780 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.524060 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.524332 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.524615 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.524890 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.525167 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.525539 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.525630 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80237) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 15:21:34.525810 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.525893 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.526097 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.526191 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.526365 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.526456 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.526636 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.526763 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.526903 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80236) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.526984 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.527181 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.527458 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.527859 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.528138 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.528431 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.528701 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.528974 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.529247 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:21:34.565540 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x557938cd29c0 pid: 80236 (default app) Jul 14 15:21:34.567684 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x557938cd29c0 pid: 80237 (default app) Jul 14 15:21:34.568970 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ac97d3-66a0-4e4d-a474-8e6cc423ca9c None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:21:34.569268 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ac97d3-66a0-4e4d-a474-8e6cc423ca9c None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:21:34.569605 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ac97d3-66a0-4e4d-a474-8e6cc423ca9c None None] PATH_INFO: `/v3` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:21:34.573830 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1/1] 10.222.0.99 () {50 vars in 791 bytes} [Thu Jul 14 15:21:34 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:21:42.354309 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dfe867-84dc-4133-9a31-23b510dbc242 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:21:42.354840 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dfe867-84dc-4133-9a31-23b510dbc242 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:21:42.354840 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dfe867-84dc-4133-9a31-23b510dbc242 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:21:42.359189 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1/2] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:21:42 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 15:21:42.365247 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf612000-5b75-4e60-8dce-1b204997e7a5 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:21:42.365665 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf612000-5b75-4e60-8dce-1b204997e7a5 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:21:42.365807 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf612000-5b75-4e60-8dce-1b204997e7a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:21:42.434752 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: INFO dbcounter [None req-bf612000-5b75-4e60-8dce-1b204997e7a5 None None] Registered counter for database keystone Jul 14 15:21:42.455914 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG oslo_db.sqlalchemy.engines [None req-bf612000-5b75-4e60-8dce-1b204997e7a5 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80236) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 14 15:21:42.462026 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writer thread running {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 14 15:21:42.640635 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG passlib.handlers.bcrypt [None req-bf612000-5b75-4e60-8dce-1b204997e7a5 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80236) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 14 15:21:42.641065 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG passlib.handlers.bcrypt [None req-bf612000-5b75-4e60-8dce-1b204997e7a5 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80236) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 14 15:21:42.696741 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-bf612000-5b75-4e60-8dce-1b204997e7a5 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:21:42.698370 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-bf612000-5b75-4e60-8dce-1b204997e7a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:21:42.826234 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 2/3] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:21:42 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 462 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jul 14 15:21:43.008564 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dfe867-84dc-4133-9a31-23b510dbc242 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:21:43.078776 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: INFO dbcounter [None req-65dfe867-84dc-4133-9a31-23b510dbc242 None None] Registered counter for database keystone Jul 14 15:21:43.098807 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG oslo_db.sqlalchemy.engines [None req-65dfe867-84dc-4133-9a31-23b510dbc242 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80237) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 14 15:21:43.104948 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writer thread running {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 14 15:21:43.224336 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dfe867-84dc-4133-9a31-23b510dbc242 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:21:43.228584 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71adca77-5f88-40c8-9b86-6bd88ae0581e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:21:43.229258 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71adca77-5f88-40c8-9b86-6bd88ae0581e admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:21:43.229547 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71adca77-5f88-40c8-9b86-6bd88ae0581e admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:21:43.229849 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71adca77-5f88-40c8-9b86-6bd88ae0581e admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:21:43.239541 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71adca77-5f88-40c8-9b86-6bd88ae0581e admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:21:43.292906 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: WARNING keystone.common.rbac_enforcer.enforcer [None req-71adca77-5f88-40c8-9b86-6bd88ae0581e admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 14 15:21:43.293174 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71adca77-5f88-40c8-9b86-6bd88ae0581e admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:21:43.304171 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 2/4] 10.222.0.99 () {58 vars in 1324 bytes} [Thu Jul 14 15:21:43 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 14 15:21:43.310386 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf612000-5b75-4e60-8dce-1b204997e7a5 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:21:43.326161 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf612000-5b75-4e60-8dce-1b204997e7a5 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:21:43.330585 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ef3eaf-ec3c-4acc-b424-94b9ea2dd7dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:21:43.332039 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ef3eaf-ec3c-4acc-b424-94b9ea2dd7dc admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:21:43.332340 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ef3eaf-ec3c-4acc-b424-94b9ea2dd7dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:21:43.332626 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ef3eaf-ec3c-4acc-b424-94b9ea2dd7dc admin admin] PATH_INFO: `/v3/projects/ae21968b232140b899e10f7d7d033a88` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:21:43.336712 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ef3eaf-ec3c-4acc-b424-94b9ea2dd7dc admin admin] RBAC: Authorizing `identity:get_project(project_id=ae21968b232140b899e10f7d7d033a88)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:21:43.388711 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: WARNING keystone.common.rbac_enforcer.enforcer [None req-a7ef3eaf-ec3c-4acc-b424-94b9ea2dd7dc admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 14 15:21:43.389300 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ef3eaf-ec3c-4acc-b424-94b9ea2dd7dc admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:21:43.394275 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 3/5] 10.222.0.99 () {56 vars in 1207 bytes} [Thu Jul 14 15:21:43 2022] GET /identity/v3/projects/ae21968b232140b899e10f7d7d033a88 => generated 344 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:21:45.471005 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98253c15-b39e-4723-acec-24cb933a3428 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:21:45.471546 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98253c15-b39e-4723-acec-24cb933a3428 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:21:45.471546 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98253c15-b39e-4723-acec-24cb933a3428 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:21:45.476326 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 3/6] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:21:45 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 15:21:45.482086 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4049d25-95b9-47a2-9ad4-9e70cbe07df3 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:21:45.482527 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4049d25-95b9-47a2-9ad4-9e70cbe07df3 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:21:45.482656 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4049d25-95b9-47a2-9ad4-9e70cbe07df3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:21:45.502047 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-a4049d25-95b9-47a2-9ad4-9e70cbe07df3 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:21:45.503659 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-a4049d25-95b9-47a2-9ad4-9e70cbe07df3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:21:45.511651 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 4/7] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:21:45 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 14 15:21:45.533969 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98253c15-b39e-4723-acec-24cb933a3428 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:21:45.546737 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98253c15-b39e-4723-acec-24cb933a3428 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:21:45.551741 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c9c165-9cd6-41a6-923b-07e76979cb0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:21:45.552386 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c9c165-9cd6-41a6-923b-07e76979cb0f admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:21:45.552674 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c9c165-9cd6-41a6-923b-07e76979cb0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:21:45.552955 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c9c165-9cd6-41a6-923b-07e76979cb0f admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:21:45.560697 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c9c165-9cd6-41a6-923b-07e76979cb0f admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:21:45.562482 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c9c165-9cd6-41a6-923b-07e76979cb0f admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:21:45.572150 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 4/8] 10.222.0.99 () {58 vars in 1324 bytes} [Thu Jul 14 15:21:45 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 15:21:45.577686 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4049d25-95b9-47a2-9ad4-9e70cbe07df3 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:21:45.587338 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4049d25-95b9-47a2-9ad4-9e70cbe07df3 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:21:45.591757 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efc526f-506a-458d-a21b-d415eadacb80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:21:45.592638 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efc526f-506a-458d-a21b-d415eadacb80 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:21:45.592925 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efc526f-506a-458d-a21b-d415eadacb80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:21:45.593202 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efc526f-506a-458d-a21b-d415eadacb80 admin admin] PATH_INFO: `/v3/users/3e33ee1d9fa74c5fa2f58e221ad2d0a8` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:21:45.594607 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efc526f-506a-458d-a21b-d415eadacb80 admin admin] RBAC: Authorizing `identity:get_user(user_id=3e33ee1d9fa74c5fa2f58e221ad2d0a8)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:21:45.596478 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efc526f-506a-458d-a21b-d415eadacb80 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:21:45.598424 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 5/9] 10.222.0.99 () {56 vars in 1198 bytes} [Thu Jul 14 15:21:45 2022] GET /identity/v3/users/3e33ee1d9fa74c5fa2f58e221ad2d0a8 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:21:47.719086 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dcc65e-ee3b-4e44-b040-199f1f6c21aa None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:21:47.719624 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dcc65e-ee3b-4e44-b040-199f1f6c21aa None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:21:47.719730 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dcc65e-ee3b-4e44-b040-199f1f6c21aa None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:21:47.721016 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 5/10] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:21:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:21:47.727307 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250c9c15-88b0-4228-ad06-b55385c7ce0f None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:21:47.727790 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250c9c15-88b0-4228-ad06-b55385c7ce0f None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:21:47.728042 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250c9c15-88b0-4228-ad06-b55385c7ce0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:21:47.745013 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-250c9c15-88b0-4228-ad06-b55385c7ce0f None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:21:47.746516 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-250c9c15-88b0-4228-ad06-b55385c7ce0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:21:47.768267 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaf4b59-9fb5-4c47-84f9-bf2887f2f434 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:21:47.768993 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaf4b59-9fb5-4c47-84f9-bf2887f2f434 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:21:47.769290 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaf4b59-9fb5-4c47-84f9-bf2887f2f434 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:21:47.771174 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 6/11] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:21:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 15:21:47.777097 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a41824d-ff05-4bc4-86e9-4381b62fbbd2 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:21:47.777097 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a41824d-ff05-4bc4-86e9-4381b62fbbd2 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:21:47.777097 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a41824d-ff05-4bc4-86e9-4381b62fbbd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:21:47.817041 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG passlib.handlers.bcrypt [None req-3a41824d-ff05-4bc4-86e9-4381b62fbbd2 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80237) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 14 15:21:47.817455 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG passlib.handlers.bcrypt [None req-3a41824d-ff05-4bc4-86e9-4381b62fbbd2 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80237) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 14 15:21:47.833131 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 6/12] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:21:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:21:47.856529 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250c9c15-88b0-4228-ad06-b55385c7ce0f None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:21:47.863480 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-3a41824d-ff05-4bc4-86e9-4381b62fbbd2 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:21:47.865141 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-3a41824d-ff05-4bc4-86e9-4381b62fbbd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:21:47.881375 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:21:47.965220 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250c9c15-88b0-4228-ad06-b55385c7ce0f None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:21:47.968882 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 7/13] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:21:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 15:21:47.992975 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a41824d-ff05-4bc4-86e9-4381b62fbbd2 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:21:48.003538 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3765cc-6a88-441d-a17f-87c506af6604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:21:48.004643 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3765cc-6a88-441d-a17f-87c506af6604 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:21:48.004754 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3765cc-6a88-441d-a17f-87c506af6604 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:21:48.004960 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3765cc-6a88-441d-a17f-87c506af6604 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:21:48.016251 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3765cc-6a88-441d-a17f-87c506af6604 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:21:48.028124 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-5c3765cc-6a88-441d-a17f-87c506af6604 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:21:48.028124 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:21:48.028124 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:21:48.028124 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:21:48.028124 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:21:48.028124 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:21:48.028124 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:21:48.028124 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:21:48.028124 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:21:48.028124 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:21:48.028124 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:21:48.028124 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:21:48.028124 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:21:48.028124 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:21:48.028124 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:21:48.028124 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:21:48.029078 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:21:48.029078 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:21:48.029078 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:21:48.029078 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:21:48.029078 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:21:48.029078 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:21:48.029078 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:21:48.029078 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:21:48.029078 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:21:48.029078 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:21:48.029078 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:21:48.029078 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:21:48.029078 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:21:48.029078 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:21:48.029078 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:21:48.029078 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:21:48.029881 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:21:48.029881 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:21:48.029881 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:21:48.029881 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:21:48.029881 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:21:48.029881 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:21:48.029881 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:21:48.029881 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:21:48.029881 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:21:48.029881 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:21:48.029881 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:21:48.029881 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:21:48.029881 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 7/14] 10.222.0.99 () {56 vars in 1096 bytes} [Thu Jul 14 15:21:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 15:21:48.036088 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3765cc-6a88-441d-a17f-87c506af6604 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:21:48.094813 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a41824d-ff05-4bc4-86e9-4381b62fbbd2 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:21:48.097139 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:21:48.118416 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:21:48.134789 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1032c47f-003f-47b0-b7aa-7e7cd76218e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:21:48.135755 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1032c47f-003f-47b0-b7aa-7e7cd76218e4 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:21:48.136110 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1032c47f-003f-47b0-b7aa-7e7cd76218e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:21:48.136399 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1032c47f-003f-47b0-b7aa-7e7cd76218e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:21:48.138307 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3765cc-6a88-441d-a17f-87c506af6604 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:21:48.172497 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1059d95-9f28-42bd-9a73-d14e639737d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:21:48.173177 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1059d95-9f28-42bd-9a73-d14e639737d2 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:21:48.173469 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1059d95-9f28-42bd-9a73-d14e639737d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:21:48.173752 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1059d95-9f28-42bd-9a73-d14e639737d2 None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:21:48.174517 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1059d95-9f28-42bd-9a73-d14e639737d2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:21:48.174833 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1059d95-9f28-42bd-9a73-d14e639737d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:21:48.176519 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1059d95-9f28-42bd-9a73-d14e639737d2 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:21:48.188273 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 8/15] 10.222.0.99 () {56 vars in 1099 bytes} [Thu Jul 14 15:21:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:21:48.194889 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1059d95-9f28-42bd-9a73-d14e639737d2 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:21:48.204580 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1032c47f-003f-47b0-b7aa-7e7cd76218e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:21:48.206120 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1032c47f-003f-47b0-b7aa-7e7cd76218e4 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:21:48.295475 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1059d95-9f28-42bd-9a73-d14e639737d2 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:21:48.303152 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 8/16] 10.222.0.99 () {58 vars in 1282 bytes} [Thu Jul 14 15:21:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 14 15:21:48.308979 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1032c47f-003f-47b0-b7aa-7e7cd76218e4 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:21:48.311795 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:21:48.330866 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d827cc9f-4432-41a0-9262-b048f3066e3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:21:48.331794 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d827cc9f-4432-41a0-9262-b048f3066e3c None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:21:48.332167 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d827cc9f-4432-41a0-9262-b048f3066e3c None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:21:48.332946 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d827cc9f-4432-41a0-9262-b048f3066e3c None admin] PATH_INFO: `/v3/users/3e33ee1d9fa74c5fa2f58e221ad2d0a8` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:21:48.334197 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d827cc9f-4432-41a0-9262-b048f3066e3c None admin] RBAC: Authorizing `identity:get_user(user_id=3e33ee1d9fa74c5fa2f58e221ad2d0a8)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:21:48.335709 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d827cc9f-4432-41a0-9262-b048f3066e3c None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:21:48.337359 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 9/17] 10.222.0.99 () {56 vars in 1177 bytes} [Thu Jul 14 15:21:48 2022] GET /identity/v3/users/3e33ee1d9fa74c5fa2f58e221ad2d0a8 => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:21:48.342746 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d827cc9f-4432-41a0-9262-b048f3066e3c None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:21:48.357139 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:21:48.410963 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1032c47f-003f-47b0-b7aa-7e7cd76218e4 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:21:48.443542 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d827cc9f-4432-41a0-9262-b048f3066e3c None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:21:48.444720 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897037ea-b0c4-40fd-9af7-1abc2deaf69f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:21:48.445529 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897037ea-b0c4-40fd-9af7-1abc2deaf69f None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:21:48.445823 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897037ea-b0c4-40fd-9af7-1abc2deaf69f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:21:48.446104 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897037ea-b0c4-40fd-9af7-1abc2deaf69f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:21:48.466988 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897037ea-b0c4-40fd-9af7-1abc2deaf69f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:21:48.468513 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897037ea-b0c4-40fd-9af7-1abc2deaf69f None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:21:48.478922 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44941d2c-a0e1-4e73-900b-b12571a92443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:21:48.479756 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44941d2c-a0e1-4e73-900b-b12571a92443 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:21:48.480048 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44941d2c-a0e1-4e73-900b-b12571a92443 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:21:48.480329 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44941d2c-a0e1-4e73-900b-b12571a92443 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:21:48.481523 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44941d2c-a0e1-4e73-900b-b12571a92443 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:21:48.482976 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44941d2c-a0e1-4e73-900b-b12571a92443 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:21:48.484610 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 10/18] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:21:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:21:48.489499 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-897037ea-b0c4-40fd-9af7-1abc2deaf69f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:21:48.489499 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:21:48.489499 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:21:48.489499 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:21:48.489499 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:21:48.489499 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:21:48.489499 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:21:48.489499 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:21:48.489499 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:21:48.489499 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:21:48.489499 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:21:48.489499 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:21:48.489499 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:21:48.489499 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:21:48.489499 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:21:48.490307 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:21:48.490307 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:21:48.490307 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:21:48.490307 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:21:48.490307 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:21:48.490307 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:21:48.490307 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:21:48.490307 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:21:48.490307 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:21:48.490307 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:21:48.490307 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:21:48.490307 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:21:48.490307 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:21:48.490307 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:21:48.490307 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:21:48.490307 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:21:48.490307 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:21:48.491093 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:21:48.491093 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:21:48.491093 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:21:48.491093 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:21:48.491297 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44941d2c-a0e1-4e73-900b-b12571a92443 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:21:48.491375 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 9/19] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:21:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 15:21:48.496727 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897037ea-b0c4-40fd-9af7-1abc2deaf69f None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:21:48.556273 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:21:48.574326 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:21:48.593564 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44941d2c-a0e1-4e73-900b-b12571a92443 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:21:48.601772 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897037ea-b0c4-40fd-9af7-1abc2deaf69f None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:21:48.628094 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fb4866-0500-4c68-9c8d-e7c2c65609f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:21:48.628759 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fb4866-0500-4c68-9c8d-e7c2c65609f5 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:21:48.629103 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fb4866-0500-4c68-9c8d-e7c2c65609f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:21:48.629879 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fb4866-0500-4c68-9c8d-e7c2c65609f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:21:48.630722 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fb4866-0500-4c68-9c8d-e7c2c65609f5 None admin] RBAC: Adding query filter params (role.id=d0f6ba1314f14d9f9c816bdb5d0c3aac, scope.domain.id=default, user.id=3e33ee1d9fa74c5fa2f58e221ad2d0a8) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:21:48.631040 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fb4866-0500-4c68-9c8d-e7c2c65609f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:21:48.632520 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fb4866-0500-4c68-9c8d-e7c2c65609f5 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:21:48.636449 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad570dd-3031-4a60-a2e6-b241ec4075e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:21:48.637118 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad570dd-3031-4a60-a2e6-b241ec4075e7 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:21:48.637408 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad570dd-3031-4a60-a2e6-b241ec4075e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:21:48.637691 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad570dd-3031-4a60-a2e6-b241ec4075e7 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:21:48.638422 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad570dd-3031-4a60-a2e6-b241ec4075e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:21:48.638734 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad570dd-3031-4a60-a2e6-b241ec4075e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:21:48.639228 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 11/20] 10.222.0.99 () {56 vars in 1322 bytes} [Thu Jul 14 15:21:48 2022] GET /identity/v3/role_assignments?user.id=3e33ee1d9fa74c5fa2f58e221ad2d0a8&scope.domain.id=default&role.id=d0f6ba1314f14d9f9c816bdb5d0c3aac => generated 233 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:21:48.640242 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad570dd-3031-4a60-a2e6-b241ec4075e7 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:21:48.660138 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 10/21] 10.222.0.99 () {56 vars in 1109 bytes} [Thu Jul 14 15:21:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:21:51.046251 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ed54d2-1902-49cb-b064-f2591f6cc864 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:21:51.047249 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ed54d2-1902-49cb-b064-f2591f6cc864 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:21:51.047870 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ed54d2-1902-49cb-b064-f2591f6cc864 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:21:51.049484 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 12/22] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:21:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:21:51.056440 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ebd6a1-cded-4136-8e6f-d9e3957cdb7a None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:21:51.057803 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ebd6a1-cded-4136-8e6f-d9e3957cdb7a None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:21:51.058610 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ebd6a1-cded-4136-8e6f-d9e3957cdb7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:21:51.081507 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-80ebd6a1-cded-4136-8e6f-d9e3957cdb7a None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:21:51.083310 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-80ebd6a1-cded-4136-8e6f-d9e3957cdb7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:21:51.084149 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c81db6-ee68-4c2e-9544-207e561488a9 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:21:51.084666 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c81db6-ee68-4c2e-9544-207e561488a9 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:21:51.085159 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c81db6-ee68-4c2e-9544-207e561488a9 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:21:51.087878 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 13/23] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:21:51 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:21:51.100957 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7566d74b-2c78-4f59-8e26-4fcf97255753 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:21:51.101723 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7566d74b-2c78-4f59-8e26-4fcf97255753 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:21:51.102213 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7566d74b-2c78-4f59-8e26-4fcf97255753 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:21:51.137517 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-7566d74b-2c78-4f59-8e26-4fcf97255753 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:21:51.139727 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-7566d74b-2c78-4f59-8e26-4fcf97255753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:21:51.242041 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 11/24] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:21:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:21:51.278997 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ebd6a1-cded-4136-8e6f-d9e3957cdb7a None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:21:51.280632 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 14/25] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:21:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:21:51.344595 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7566d74b-2c78-4f59-8e26-4fcf97255753 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:21:51.375641 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:21:51.518131 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ebd6a1-cded-4136-8e6f-d9e3957cdb7a None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:21:51.541589 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:21:51.610138 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7566d74b-2c78-4f59-8e26-4fcf97255753 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:21:51.626594 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46f7eb7-16b4-4ca6-a2cc-38a47009abbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:21:51.627682 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46f7eb7-16b4-4ca6-a2cc-38a47009abbe None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:21:51.628144 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46f7eb7-16b4-4ca6-a2cc-38a47009abbe None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:21:51.628574 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46f7eb7-16b4-4ca6-a2cc-38a47009abbe None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:21:51.658736 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46f7eb7-16b4-4ca6-a2cc-38a47009abbe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:21:51.661970 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fabf593-1570-4b48-9d7c-1b282428d447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:21:51.662951 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fabf593-1570-4b48-9d7c-1b282428d447 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:21:51.663407 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fabf593-1570-4b48-9d7c-1b282428d447 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:21:51.663860 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fabf593-1570-4b48-9d7c-1b282428d447 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:21:51.679630 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-b46f7eb7-16b4-4ca6-a2cc-38a47009abbe None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:21:51.679630 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:21:51.679630 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:21:51.679630 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:21:51.679630 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:21:51.679630 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:21:51.679630 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:21:51.679630 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:21:51.679630 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:21:51.679630 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:21:51.679630 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:21:51.679630 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:21:51.679630 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:21:51.679630 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:21:51.679630 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:21:51.679630 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:21:51.680737 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:21:51.680737 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:21:51.680737 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:21:51.680737 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:21:51.680737 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:21:51.680737 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:21:51.680737 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:21:51.680737 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:21:51.680737 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:21:51.680737 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:21:51.680737 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:21:51.680737 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:21:51.680737 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:21:51.680737 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:21:51.680737 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:21:51.680737 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:21:51.681689 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:21:51.681689 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:21:51.681689 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:21:51.681689 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:21:51.681689 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:21:51.681689 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:21:51.681689 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:21:51.681689 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:21:51.681689 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:21:51.681689 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:21:51.681689 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:21:51.681689 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:21:51.684257 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 12/26] 10.222.0.99 () {56 vars in 1096 bytes} [Thu Jul 14 15:21:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 406 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:21:51.687788 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc10c67-fff3-4d01-b739-6221aa98a5cd None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:21:51.688340 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc10c67-fff3-4d01-b739-6221aa98a5cd None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:21:51.690298 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc10c67-fff3-4d01-b739-6221aa98a5cd None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:21:51.691823 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 13/27] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:21:51 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:21:51.693747 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fabf593-1570-4b48-9d7c-1b282428d447 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:21:51.695574 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fabf593-1570-4b48-9d7c-1b282428d447 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:21:51.696480 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc10c67-fff3-4d01-b739-6221aa98a5cd None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:21:51.731509 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-0fabf593-1570-4b48-9d7c-1b282428d447 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:21:51.731509 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:21:51.731509 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:21:51.731509 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:21:51.731509 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:21:51.731509 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:21:51.731509 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:21:51.731509 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:21:51.731509 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:21:51.731509 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:21:51.731509 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:21:51.731509 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:21:51.731509 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:21:51.731509 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:21:51.731509 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:21:51.732592 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:21:51.732592 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:21:51.732592 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:21:51.732592 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:21:51.732592 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:21:51.732592 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:21:51.732592 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:21:51.732592 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:21:51.732592 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:21:51.732592 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:21:51.732592 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:21:51.732592 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:21:51.732592 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:21:51.732592 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:21:51.732592 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:21:51.732592 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:21:51.732592 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:21:51.733586 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:21:51.733586 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:21:51.733586 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:21:51.733586 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:21:51.750403 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 15/28] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:21:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 401 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:21:51.750403 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029bb845-6356-445a-81be-f140d41bafc9 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:21:51.750403 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029bb845-6356-445a-81be-f140d41bafc9 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:21:51.750403 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029bb845-6356-445a-81be-f140d41bafc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:21:51.778182 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-029bb845-6356-445a-81be-f140d41bafc9 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:21:51.781130 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-029bb845-6356-445a-81be-f140d41bafc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:21:51.841343 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc10c67-fff3-4d01-b739-6221aa98a5cd None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:21:51.850072 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:21:51.857537 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 16/29] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:21:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:21:51.858644 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029bb845-6356-445a-81be-f140d41bafc9 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:21:51.881279 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429d4ec4-1df7-470e-993e-c6ed96c8aa79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:21:51.882079 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429d4ec4-1df7-470e-993e-c6ed96c8aa79 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:21:51.882571 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429d4ec4-1df7-470e-993e-c6ed96c8aa79 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:21:51.882994 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429d4ec4-1df7-470e-993e-c6ed96c8aa79 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:21:51.883963 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429d4ec4-1df7-470e-993e-c6ed96c8aa79 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:21:51.884421 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429d4ec4-1df7-470e-993e-c6ed96c8aa79 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:21:51.886308 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429d4ec4-1df7-470e-993e-c6ed96c8aa79 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:21:51.903493 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 14/30] 10.222.0.99 () {56 vars in 1099 bytes} [Thu Jul 14 15:21:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:21:51.906011 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4227ce35-469b-4b47-839a-75d1089d7c67 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:21:51.906496 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4227ce35-469b-4b47-839a-75d1089d7c67 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:21:51.907041 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4227ce35-469b-4b47-839a-75d1089d7c67 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:21:51.908684 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 15/31] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:21:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:21:51.909823 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4227ce35-469b-4b47-839a-75d1089d7c67 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:21:51.925149 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:21:51.983992 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029bb845-6356-445a-81be-f140d41bafc9 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:21:52.026041 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec768b4-239f-4054-8805-bc1463e98acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:21:52.027177 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec768b4-239f-4054-8805-bc1463e98acc None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:21:52.027704 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec768b4-239f-4054-8805-bc1463e98acc None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:21:52.028191 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec768b4-239f-4054-8805-bc1463e98acc None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:21:52.029177 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec768b4-239f-4054-8805-bc1463e98acc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:21:52.029688 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec768b4-239f-4054-8805-bc1463e98acc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:21:52.031295 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec768b4-239f-4054-8805-bc1463e98acc None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:21:52.041071 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4227ce35-469b-4b47-839a-75d1089d7c67 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:21:52.051556 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 17/32] 10.222.0.99 () {56 vars in 1109 bytes} [Thu Jul 14 15:21:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:21:52.057724 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec768b4-239f-4054-8805-bc1463e98acc None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:21:52.078866 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f1efd6-8ff4-4524-b493-35fbaa503dd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:21:52.079542 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f1efd6-8ff4-4524-b493-35fbaa503dd0 None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:21:52.079950 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f1efd6-8ff4-4524-b493-35fbaa503dd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:21:52.080242 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f1efd6-8ff4-4524-b493-35fbaa503dd0 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:21:52.081057 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f1efd6-8ff4-4524-b493-35fbaa503dd0 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:21:52.082535 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f1efd6-8ff4-4524-b493-35fbaa503dd0 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:21:52.171231 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec768b4-239f-4054-8805-bc1463e98acc None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:21:52.189258 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:21:52.211206 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556ba174-819f-441e-a12b-9308ef15cc7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:21:52.212508 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556ba174-819f-441e-a12b-9308ef15cc7e None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:21:52.213003 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556ba174-819f-441e-a12b-9308ef15cc7e None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:21:52.213499 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556ba174-819f-441e-a12b-9308ef15cc7e None admin] PATH_INFO: `/v3/users/3e33ee1d9fa74c5fa2f58e221ad2d0a8` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:21:52.214958 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556ba174-819f-441e-a12b-9308ef15cc7e None admin] RBAC: Authorizing `identity:get_user(user_id=3e33ee1d9fa74c5fa2f58e221ad2d0a8)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:21:52.216071 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 16/33] 10.222.0.99 () {60 vars in 1131 bytes} [Thu Jul 14 15:21:51 2022] POST /identity/v3/roles => generated 219 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:21:52.217001 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556ba174-819f-441e-a12b-9308ef15cc7e None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:21:52.217699 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa28647-0ae0-459e-850c-ac43dcee32ae None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:21:52.218007 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa28647-0ae0-459e-850c-ac43dcee32ae None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:21:52.218299 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa28647-0ae0-459e-850c-ac43dcee32ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:21:52.219638 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 18/34] 10.222.0.99 () {56 vars in 1177 bytes} [Thu Jul 14 15:21:52 2022] GET /identity/v3/users/3e33ee1d9fa74c5fa2f58e221ad2d0a8 => generated 247 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:21:52.220841 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556ba174-819f-441e-a12b-9308ef15cc7e None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:21:52.239560 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-1fa28647-0ae0-459e-850c-ac43dcee32ae None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:21:52.240947 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-1fa28647-0ae0-459e-850c-ac43dcee32ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:21:52.280240 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:21:52.335431 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 17/35] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:21:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:21:52.336240 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa28647-0ae0-459e-850c-ac43dcee32ae None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:21:52.419438 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556ba174-819f-441e-a12b-9308ef15cc7e None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:21:52.447360 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa28647-0ae0-459e-850c-ac43dcee32ae None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:21:52.461000 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f4add1-dd06-42cc-9061-661e0a3150e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:21:52.462662 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f4add1-dd06-42cc-9061-661e0a3150e9 None admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:21:52.463163 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f4add1-dd06-42cc-9061-661e0a3150e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:21:52.463714 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f4add1-dd06-42cc-9061-661e0a3150e9 None admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:21:52.464753 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f4add1-dd06-42cc-9061-661e0a3150e9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:21:52.466434 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f4add1-dd06-42cc-9061-661e0a3150e9 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:21:52.484800 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e53a3eb-5edf-43c0-a65c-c77c4dc50fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:21:52.485622 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e53a3eb-5edf-43c0-a65c-c77c4dc50fe3 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:21:52.486064 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e53a3eb-5edf-43c0-a65c-c77c4dc50fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:21:52.486357 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e53a3eb-5edf-43c0-a65c-c77c4dc50fe3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:21:52.489318 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e53a3eb-5edf-43c0-a65c-c77c4dc50fe3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:21:52.490487 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e53a3eb-5edf-43c0-a65c-c77c4dc50fe3 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:21:52.492710 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 18/36] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:21:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:21:52.493330 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e53a3eb-5edf-43c0-a65c-c77c4dc50fe3 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:21:52.524696 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:21:52.608861 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e53a3eb-5edf-43c0-a65c-c77c4dc50fe3 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:21:52.619044 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 19/37] 10.222.0.99 () {60 vars in 1141 bytes} [Thu Jul 14 15:21:52 2022] POST /identity/v3/projects => generated 301 bytes in 399 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:21:52.621035 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f4add1-dd06-42cc-9061-661e0a3150e9 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:21:52.653939 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc162309-1b8d-4f7d-8992-016589c86b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:21:52.654691 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc162309-1b8d-4f7d-8992-016589c86b92 None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:21:52.654987 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc162309-1b8d-4f7d-8992-016589c86b92 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:21:52.655274 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc162309-1b8d-4f7d-8992-016589c86b92 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:21:52.656851 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc162309-1b8d-4f7d-8992-016589c86b92 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:21:52.658051 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc162309-1b8d-4f7d-8992-016589c86b92 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:21:52.679696 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 19/38] 10.222.0.99 () {60 vars in 1131 bytes} [Thu Jul 14 15:21:52 2022] POST /identity/v3/roles => generated 225 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:21:52.681234 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da840cb1-62c6-4365-9b36-ac82423cc344 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:21:52.681699 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da840cb1-62c6-4365-9b36-ac82423cc344 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:21:52.682736 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da840cb1-62c6-4365-9b36-ac82423cc344 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:21:52.684017 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:21:52.688280 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 20/39] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:21:52 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:21:52.697813 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de3d6f7-207c-428e-b37a-d43c84625c6e None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:21:52.698622 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de3d6f7-207c-428e-b37a-d43c84625c6e None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:21:52.699492 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de3d6f7-207c-428e-b37a-d43c84625c6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:21:52.718279 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-3de3d6f7-207c-428e-b37a-d43c84625c6e None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:21:52.719629 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-3de3d6f7-207c-428e-b37a-d43c84625c6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:21:52.772368 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f4add1-dd06-42cc-9061-661e0a3150e9 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:21:52.801322 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 21/40] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:21:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:21:52.811782 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3a5bc2-d7de-4fd0-b170-bd694d6ad4eb None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:21:52.812206 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3a5bc2-d7de-4fd0-b170-bd694d6ad4eb None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:21:52.812503 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3a5bc2-d7de-4fd0-b170-bd694d6ad4eb None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:21:52.814047 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 22/41] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:21:52 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:21:52.818647 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857ba043-d584-4166-80d2-632c0139af3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:21:52.819733 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8075f47b-a289-49d6-9749-ade5eada743f None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:21:52.820043 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8075f47b-a289-49d6-9749-ade5eada743f None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:21:52.820338 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8075f47b-a289-49d6-9749-ade5eada743f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:21:52.820822 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857ba043-d584-4166-80d2-632c0139af3b None admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:21:52.821320 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857ba043-d584-4166-80d2-632c0139af3b None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:21:52.821794 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857ba043-d584-4166-80d2-632c0139af3b None admin] PATH_INFO: `/v3/domains/default/users/3e33ee1d9fa74c5fa2f58e221ad2d0a8/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:21:52.824548 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857ba043-d584-4166-80d2-632c0139af3b None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=3e33ee1d9fa74c5fa2f58e221ad2d0a8, role_id=d0f6ba1314f14d9f9c816bdb5d0c3aac)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:21:52.826442 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857ba043-d584-4166-80d2-632c0139af3b None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:21:52.845181 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 20/42] 10.222.0.99 () {58 vars in 1361 bytes} [Thu Jul 14 15:21:52 2022] PUT /identity/v3/domains/default/users/3e33ee1d9fa74c5fa2f58e221ad2d0a8/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:21:52.846629 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857ba043-d584-4166-80d2-632c0139af3b None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:21:52.851539 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-8075f47b-a289-49d6-9749-ade5eada743f None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:21:52.853091 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-8075f47b-a289-49d6-9749-ade5eada743f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:21:52.873614 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:21:52.972323 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 23/43] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:21:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 15:21:53.012432 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8075f47b-a289-49d6-9749-ade5eada743f None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:21:53.113956 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:21:53.122085 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857ba043-d584-4166-80d2-632c0139af3b None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:21:53.148618 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8075f47b-a289-49d6-9749-ade5eada743f None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:21:53.162043 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad84f7e2-89ac-4a14-9e1a-e9ca045b7b2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:21:53.162739 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad84f7e2-89ac-4a14-9e1a-e9ca045b7b2c None admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:21:53.163034 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad84f7e2-89ac-4a14-9e1a-e9ca045b7b2c None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:21:53.163430 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad84f7e2-89ac-4a14-9e1a-e9ca045b7b2c None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:21:53.164520 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad84f7e2-89ac-4a14-9e1a-e9ca045b7b2c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:21:53.165977 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad84f7e2-89ac-4a14-9e1a-e9ca045b7b2c None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:21:53.181344 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 21/44] 10.222.0.99 () {60 vars in 1131 bytes} [Thu Jul 14 15:21:52 2022] POST /identity/v3/roles => generated 223 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:21:53.196207 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97283bff-7598-49ce-83b8-bd06453f7bfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:21:53.197416 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97283bff-7598-49ce-83b8-bd06453f7bfb None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:21:53.197919 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97283bff-7598-49ce-83b8-bd06453f7bfb None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:21:53.198393 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97283bff-7598-49ce-83b8-bd06453f7bfb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:21:53.200024 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97283bff-7598-49ce-83b8-bd06453f7bfb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:21:53.201824 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97283bff-7598-49ce-83b8-bd06453f7bfb None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:21:53.204373 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 24/45] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:21:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:21:53.209975 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad84f7e2-89ac-4a14-9e1a-e9ca045b7b2c None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:21:53.331928 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad84f7e2-89ac-4a14-9e1a-e9ca045b7b2c None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:21:53.377632 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de60279-2eb1-43ff-aee7-c86e2110168e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:21:53.378312 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de60279-2eb1-43ff-aee7-c86e2110168e None admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:21:53.378612 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de60279-2eb1-43ff-aee7-c86e2110168e None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:21:53.378900 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de60279-2eb1-43ff-aee7-c86e2110168e None admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:21:53.379706 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de60279-2eb1-43ff-aee7-c86e2110168e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:21:53.382048 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de60279-2eb1-43ff-aee7-c86e2110168e None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:21:53.419666 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 22/46] 10.222.0.99 () {60 vars in 1141 bytes} [Thu Jul 14 15:21:53 2022] POST /identity/v3/projects => generated 312 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:21:54.984263 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdff502f-39af-44c0-987c-252f59b124c4 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:21:54.984777 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdff502f-39af-44c0-987c-252f59b124c4 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:21:54.984862 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdff502f-39af-44c0-987c-252f59b124c4 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:21:54.986219 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 25/47] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:21:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:21:54.992547 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b7a011-dd23-422e-aa4f-5f14bf8809fe None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:21:54.993010 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b7a011-dd23-422e-aa4f-5f14bf8809fe None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:21:54.993151 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b7a011-dd23-422e-aa4f-5f14bf8809fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:21:55.010889 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-69b7a011-dd23-422e-aa4f-5f14bf8809fe None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:21:55.012458 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-69b7a011-dd23-422e-aa4f-5f14bf8809fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:21:55.014525 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:21:55.081041 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 23/48] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:21:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:21:55.105174 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdff502f-39af-44c0-987c-252f59b124c4 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:21:55.121235 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:21:55.209765 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdff502f-39af-44c0-987c-252f59b124c4 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:21:55.243108 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1748f9b7-a1f8-43ab-90e3-c081914dad8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:21:55.243863 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1748f9b7-a1f8-43ab-90e3-c081914dad8f None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:21:55.244149 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1748f9b7-a1f8-43ab-90e3-c081914dad8f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:21:55.244425 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1748f9b7-a1f8-43ab-90e3-c081914dad8f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:21:55.254765 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1748f9b7-a1f8-43ab-90e3-c081914dad8f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:21:55.262899 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-1748f9b7-a1f8-43ab-90e3-c081914dad8f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:21:55.262899 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:21:55.262899 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:21:55.262899 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:21:55.262899 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:21:55.262899 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:21:55.262899 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:21:55.262899 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:21:55.262899 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:21:55.262899 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:21:55.262899 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:21:55.262899 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:21:55.262899 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:21:55.262899 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:21:55.262899 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:21:55.262899 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:21:55.263817 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:21:55.263817 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:21:55.263817 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:21:55.263817 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:21:55.263817 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:21:55.263817 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:21:55.263817 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:21:55.263817 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:21:55.263817 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:21:55.263817 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:21:55.263817 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:21:55.263817 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:21:55.263817 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:21:55.263817 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:21:55.263817 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:21:55.263817 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:21:55.264713 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:21:55.264713 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:21:55.264713 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:21:55.264713 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:21:55.264713 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:21:55.264713 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:21:55.264713 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:21:55.264713 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:21:55.264713 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:21:55.264713 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:21:55.264713 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:21:55.264713 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:21:55.264713 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 26/49] 10.222.0.99 () {56 vars in 1096 bytes} [Thu Jul 14 15:21:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:21:55.270302 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b7a011-dd23-422e-aa4f-5f14bf8809fe None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:21:55.368575 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b7a011-dd23-422e-aa4f-5f14bf8809fe None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:21:55.402834 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cd7cca-de05-4027-aa3a-90c539bec07a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:21:55.403230 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cd7cca-de05-4027-aa3a-90c539bec07a None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:21:55.403518 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cd7cca-de05-4027-aa3a-90c539bec07a None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:21:55.403827 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cd7cca-de05-4027-aa3a-90c539bec07a None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:21:55.404581 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cd7cca-de05-4027-aa3a-90c539bec07a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:21:55.404890 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cd7cca-de05-4027-aa3a-90c539bec07a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:21:55.406289 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cd7cca-de05-4027-aa3a-90c539bec07a None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:21:55.411364 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:21:55.415781 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 24/50] 10.222.0.99 () {56 vars in 1099 bytes} [Thu Jul 14 15:21:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:21:55.421522 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1748f9b7-a1f8-43ab-90e3-c081914dad8f None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:21:55.486023 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defb6708-48b9-4711-8a59-6aed276511ce None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:21:55.486322 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defb6708-48b9-4711-8a59-6aed276511ce None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:21:55.486939 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defb6708-48b9-4711-8a59-6aed276511ce None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:21:55.488362 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 25/51] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:21:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 15:21:55.493602 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cd564b-3779-43f8-940f-50fc6ccb39dd None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:21:55.493836 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cd564b-3779-43f8-940f-50fc6ccb39dd None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:21:55.494124 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cd564b-3779-43f8-940f-50fc6ccb39dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:21:55.496594 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:21:55.511800 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-a1cd564b-3779-43f8-940f-50fc6ccb39dd None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:21:55.515112 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-a1cd564b-3779-43f8-940f-50fc6ccb39dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:21:55.523377 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1748f9b7-a1f8-43ab-90e3-c081914dad8f None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:21:55.558777 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de22a6c4-c2a9-40f9-8db3-4f691664730a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:21:55.559721 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de22a6c4-c2a9-40f9-8db3-4f691664730a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:21:55.560088 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de22a6c4-c2a9-40f9-8db3-4f691664730a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:21:55.560377 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de22a6c4-c2a9-40f9-8db3-4f691664730a None admin] PATH_INFO: `/v3/users/3e33ee1d9fa74c5fa2f58e221ad2d0a8` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:21:55.561838 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de22a6c4-c2a9-40f9-8db3-4f691664730a None admin] RBAC: Authorizing `identity:get_user(user_id=3e33ee1d9fa74c5fa2f58e221ad2d0a8)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:21:55.563301 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de22a6c4-c2a9-40f9-8db3-4f691664730a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:21:55.564993 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 27/52] 10.222.0.99 () {56 vars in 1177 bytes} [Thu Jul 14 15:21:55 2022] GET /identity/v3/users/3e33ee1d9fa74c5fa2f58e221ad2d0a8 => generated 247 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:21:55.570816 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de22a6c4-c2a9-40f9-8db3-4f691664730a None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:21:55.581575 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 26/53] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:21:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 15:21:55.605694 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cd564b-3779-43f8-940f-50fc6ccb39dd None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:21:55.714712 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cd564b-3779-43f8-940f-50fc6ccb39dd None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:21:55.745502 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:21:55.752110 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5f4736-f2ac-4bc0-aed1-57afb53f9953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:21:55.752927 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5f4736-f2ac-4bc0-aed1-57afb53f9953 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:21:55.753218 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5f4736-f2ac-4bc0-aed1-57afb53f9953 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:21:55.753502 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5f4736-f2ac-4bc0-aed1-57afb53f9953 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:21:55.754716 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5f4736-f2ac-4bc0-aed1-57afb53f9953 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:21:55.756834 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5f4736-f2ac-4bc0-aed1-57afb53f9953 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:21:55.758476 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 27/54] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:21:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:21:55.764209 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5f4736-f2ac-4bc0-aed1-57afb53f9953 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:21:55.779237 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de22a6c4-c2a9-40f9-8db3-4f691664730a None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:21:55.814016 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9034008-4096-4b55-a502-8d0bd27e51d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:21:55.814813 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9034008-4096-4b55-a502-8d0bd27e51d1 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:21:55.815099 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9034008-4096-4b55-a502-8d0bd27e51d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:21:55.815378 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9034008-4096-4b55-a502-8d0bd27e51d1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:21:55.816580 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9034008-4096-4b55-a502-8d0bd27e51d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:21:55.818626 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9034008-4096-4b55-a502-8d0bd27e51d1 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:21:55.820708 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 28/55] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:21:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:21:55.826182 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9034008-4096-4b55-a502-8d0bd27e51d1 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:21:55.839228 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:21:55.869485 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5f4736-f2ac-4bc0-aed1-57afb53f9953 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:21:55.905525 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e7da17-5360-446b-9753-7bc7f82bb14f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:21:55.906202 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e7da17-5360-446b-9753-7bc7f82bb14f None admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:21:55.906499 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e7da17-5360-446b-9753-7bc7f82bb14f None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:21:55.906779 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e7da17-5360-446b-9753-7bc7f82bb14f None admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:21:55.907496 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e7da17-5360-446b-9753-7bc7f82bb14f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:21:55.908973 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e7da17-5360-446b-9753-7bc7f82bb14f None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:21:55.929303 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9034008-4096-4b55-a502-8d0bd27e51d1 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:21:55.934207 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 28/56] 10.222.0.99 () {60 vars in 1140 bytes} [Thu Jul 14 15:21:55 2022] POST /identity/v3/projects => generated 298 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 15:21:55.966142 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011dd385-1e0e-4bc3-8942-e2a383a8fabb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:21:55.966832 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011dd385-1e0e-4bc3-8942-e2a383a8fabb None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:21:55.967129 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011dd385-1e0e-4bc3-8942-e2a383a8fabb None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:21:55.967416 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011dd385-1e0e-4bc3-8942-e2a383a8fabb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:21:55.968331 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011dd385-1e0e-4bc3-8942-e2a383a8fabb None admin] RBAC: Adding query filter params (role.id=d0f6ba1314f14d9f9c816bdb5d0c3aac, scope.domain.id=default, user.id=3e33ee1d9fa74c5fa2f58e221ad2d0a8) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:21:55.968656 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011dd385-1e0e-4bc3-8942-e2a383a8fabb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:21:55.970175 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011dd385-1e0e-4bc3-8942-e2a383a8fabb None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:21:55.980595 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 29/57] 10.222.0.99 () {56 vars in 1322 bytes} [Thu Jul 14 15:21:55 2022] GET /identity/v3/role_assignments?user.id=3e33ee1d9fa74c5fa2f58e221ad2d0a8&scope.domain.id=default&role.id=d0f6ba1314f14d9f9c816bdb5d0c3aac => generated 530 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:21:58.026920 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c28f20-6b48-48f9-9ac8-7f4a897fa080 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:21:58.027358 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c28f20-6b48-48f9-9ac8-7f4a897fa080 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:21:58.027505 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c28f20-6b48-48f9-9ac8-7f4a897fa080 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:21:58.029015 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 29/58] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:21:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:21:58.034221 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48268bf0-5b5a-4b83-ac13-ab032300c6f7 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:21:58.034616 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48268bf0-5b5a-4b83-ac13-ab032300c6f7 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:21:58.034805 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48268bf0-5b5a-4b83-ac13-ab032300c6f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:21:58.052111 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-48268bf0-5b5a-4b83-ac13-ab032300c6f7 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:21:58.054284 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-48268bf0-5b5a-4b83-ac13-ab032300c6f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:21:58.120555 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 30/59] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:21:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:21:58.145120 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c28f20-6b48-48f9-9ac8-7f4a897fa080 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:21:58.197525 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:21:58.250195 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c28f20-6b48-48f9-9ac8-7f4a897fa080 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:21:58.283780 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2101f48c-17f1-45a6-8483-1d7f756b5b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:21:58.284583 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2101f48c-17f1-45a6-8483-1d7f756b5b18 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:21:58.284875 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2101f48c-17f1-45a6-8483-1d7f756b5b18 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:21:58.285160 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2101f48c-17f1-45a6-8483-1d7f756b5b18 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:21:58.286352 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2101f48c-17f1-45a6-8483-1d7f756b5b18 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:21:58.288413 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2101f48c-17f1-45a6-8483-1d7f756b5b18 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:21:58.290026 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 30/60] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:21:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:21:58.295580 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48268bf0-5b5a-4b83-ac13-ab032300c6f7 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:21:58.298710 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:21:58.394808 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48268bf0-5b5a-4b83-ac13-ab032300c6f7 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:21:58.433231 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d835fe-8870-40dc-8763-94e87274dfb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:21:58.433904 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d835fe-8870-40dc-8763-94e87274dfb8 None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:21:58.434199 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d835fe-8870-40dc-8763-94e87274dfb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:21:58.434484 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d835fe-8870-40dc-8763-94e87274dfb8 None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:21:58.435217 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d835fe-8870-40dc-8763-94e87274dfb8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:21:58.436713 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d835fe-8870-40dc-8763-94e87274dfb8 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:21:58.462515 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 31/61] 10.222.0.99 () {60 vars in 1132 bytes} [Thu Jul 14 15:21:58 2022] POST /identity/v3/users => generated 275 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 15:22:01.447247 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7497b0-54af-4ba7-996e-b1c67bf7cf69 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:01.448161 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7497b0-54af-4ba7-996e-b1c67bf7cf69 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:01.448598 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7497b0-54af-4ba7-996e-b1c67bf7cf69 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:01.450376 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 31/62] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:22:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:22:01.456597 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafb873f-2dee-44d8-9200-1dbbed176458 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:01.457401 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafb873f-2dee-44d8-9200-1dbbed176458 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:01.457854 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafb873f-2dee-44d8-9200-1dbbed176458 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:01.491521 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-eafb873f-2dee-44d8-9200-1dbbed176458 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:22:01.493429 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-eafb873f-2dee-44d8-9200-1dbbed176458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:22:01.575491 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:01.591274 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 32/63] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:22:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:22:01.626778 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7497b0-54af-4ba7-996e-b1c67bf7cf69 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:01.775735 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7497b0-54af-4ba7-996e-b1c67bf7cf69 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:01.797730 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:01.825192 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1576743-c986-40e5-b497-0d7f31daa392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:01.826620 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1576743-c986-40e5-b497-0d7f31daa392 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:01.826926 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1576743-c986-40e5-b497-0d7f31daa392 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:01.827828 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1576743-c986-40e5-b497-0d7f31daa392 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:01.844721 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1576743-c986-40e5-b497-0d7f31daa392 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:22:01.853354 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-b1576743-c986-40e5-b497-0d7f31daa392 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 15:22:01.853354 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:22:01.853354 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:22:01.853354 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:22:01.853354 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:22:01.853354 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:22:01.853354 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:22:01.853354 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:22:01.853354 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:22:01.853354 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:22:01.853354 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:22:01.853354 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:22:01.853354 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:22:01.853354 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:22:01.853354 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:22:01.853354 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:22:01.854220 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:22:01.854220 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:22:01.854220 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:22:01.854220 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:22:01.854220 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:22:01.854220 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:22:01.854220 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:22:01.854220 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:22:01.854220 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:22:01.854220 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:22:01.854220 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:22:01.854220 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:22:01.854220 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:22:01.854220 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:22:01.854220 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:22:01.854220 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:22:01.854980 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:22:01.854980 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:22:01.854980 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:22:01.854980 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:22:01.854980 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:22:01.854980 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:22:01.854980 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:22:01.854980 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:22:01.854980 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:22:01.854980 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:22:01.854980 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 15:22:01.854980 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:22:01.854980 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 32/64] 10.222.0.99 () {56 vars in 1099 bytes} [Thu Jul 14 15:22:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:22:01.862224 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafb873f-2dee-44d8-9200-1dbbed176458 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:01.868707 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ab1256-ebdf-402c-b7e8-c7285b8d0859 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:01.869003 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ab1256-ebdf-402c-b7e8-c7285b8d0859 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:01.869287 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ab1256-ebdf-402c-b7e8-c7285b8d0859 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:01.870794 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 33/65] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:22:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 15:22:01.880945 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736454a7-83b1-436c-8836-8b915dc4b016 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:01.881231 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736454a7-83b1-436c-8836-8b915dc4b016 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:01.882354 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736454a7-83b1-436c-8836-8b915dc4b016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:01.912676 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-736454a7-83b1-436c-8836-8b915dc4b016 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:22:01.914455 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-736454a7-83b1-436c-8836-8b915dc4b016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:22:02.025318 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafb873f-2dee-44d8-9200-1dbbed176458 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:02.044249 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 34/66] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:22:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 15:22:02.055622 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ad6057-ca2a-4919-b9e9-f03f6f64ee4a None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:02.055622 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ad6057-ca2a-4919-b9e9-f03f6f64ee4a None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:02.055622 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ad6057-ca2a-4919-b9e9-f03f6f64ee4a None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:02.055622 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 35/67] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:22:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:22:02.061653 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381af8fe-4b92-4de0-a7b4-181b7698f633 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:02.061950 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381af8fe-4b92-4de0-a7b4-181b7698f633 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:02.062241 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381af8fe-4b92-4de0-a7b4-181b7698f633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:02.078227 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503f7973-f957-4438-9b59-1e07a1509d1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:02.078916 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503f7973-f957-4438-9b59-1e07a1509d1f None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:02.079212 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503f7973-f957-4438-9b59-1e07a1509d1f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:02.079504 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503f7973-f957-4438-9b59-1e07a1509d1f None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:02.080409 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503f7973-f957-4438-9b59-1e07a1509d1f None admin] RBAC: Adding query filter params (name=member) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:22:02.080674 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-381af8fe-4b92-4de0-a7b4-181b7698f633 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:22:02.080779 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503f7973-f957-4438-9b59-1e07a1509d1f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:02.081888 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-381af8fe-4b92-4de0-a7b4-181b7698f633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:22:02.082112 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503f7973-f957-4438-9b59-1e07a1509d1f None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:02.091872 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 33/68] 10.222.0.99 () {56 vars in 1101 bytes} [Thu Jul 14 15:22:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:02.092673 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503f7973-f957-4438-9b59-1e07a1509d1f None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:02.158113 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:02.160458 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 36/69] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:22:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 15:22:02.161411 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381af8fe-4b92-4de0-a7b4-181b7698f633 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:02.213057 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:02.214184 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503f7973-f957-4438-9b59-1e07a1509d1f None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:02.252609 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a603c1-3590-4f6e-8bbb-a8b4944e6ccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:02.253412 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a603c1-3590-4f6e-8bbb-a8b4944e6ccb None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:02.253703 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a603c1-3590-4f6e-8bbb-a8b4944e6ccb None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:02.253989 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a603c1-3590-4f6e-8bbb-a8b4944e6ccb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:02.269354 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a603c1-3590-4f6e-8bbb-a8b4944e6ccb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:22:02.278075 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-58a603c1-3590-4f6e-8bbb-a8b4944e6ccb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:22:02.278075 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:22:02.278075 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:22:02.278075 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:22:02.278075 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:22:02.278075 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:22:02.278075 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:22:02.278075 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:22:02.278075 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:22:02.278075 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:22:02.278075 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:22:02.278075 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:22:02.278075 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:22:02.278075 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:22:02.278075 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:22:02.278075 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:22:02.278929 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:22:02.278929 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:22:02.278929 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:22:02.278929 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:22:02.278929 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:22:02.278929 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:22:02.278929 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:22:02.278929 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:22:02.278929 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:22:02.278929 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:22:02.278929 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:22:02.278929 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:22:02.278929 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:22:02.278929 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:22:02.278929 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:22:02.278929 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:22:02.280111 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:22:02.280111 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:22:02.280111 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:22:02.280111 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:22:02.280111 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:22:02.280111 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:22:02.280111 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:22:02.280111 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:22:02.280111 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:22:02.280111 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:22:02.280111 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:22:02.280111 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:22:02.280111 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 34/70] 10.222.0.99 () {56 vars in 1096 bytes} [Thu Jul 14 15:22:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:22:02.280945 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae27b183-51d2-4617-97f4-c212bd1ae3ee None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:02.280945 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae27b183-51d2-4617-97f4-c212bd1ae3ee None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:02.281204 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae27b183-51d2-4617-97f4-c212bd1ae3ee None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:02.282570 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 35/71] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:22:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:22:02.284350 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381af8fe-4b92-4de0-a7b4-181b7698f633 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:02.287063 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae27b183-51d2-4617-97f4-c212bd1ae3ee None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:02.322501 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86751f43-06d3-404c-bc19-47eb9daa8521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:02.323353 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86751f43-06d3-404c-bc19-47eb9daa8521 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:02.323666 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86751f43-06d3-404c-bc19-47eb9daa8521 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:02.323994 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86751f43-06d3-404c-bc19-47eb9daa8521 None admin] PATH_INFO: `/v3/users/7f5177fcb1524b918c6662c934105643` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:02.339418 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86751f43-06d3-404c-bc19-47eb9daa8521 None admin] RBAC: Authorizing `identity:get_user(user_id=7f5177fcb1524b918c6662c934105643)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:02.341144 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86751f43-06d3-404c-bc19-47eb9daa8521 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:02.343052 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 37/72] 10.222.0.99 () {56 vars in 1177 bytes} [Thu Jul 14 15:22:02 2022] GET /identity/v3/users/7f5177fcb1524b918c6662c934105643 => generated 275 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:02.343633 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86751f43-06d3-404c-bc19-47eb9daa8521 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:02.409496 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae27b183-51d2-4617-97f4-c212bd1ae3ee None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:02.423448 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:02.447248 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5060674-41c1-45c8-80ce-a96f3cda7ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:02.448372 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5060674-41c1-45c8-80ce-a96f3cda7ae6 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:02.448656 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5060674-41c1-45c8-80ce-a96f3cda7ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:02.448948 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5060674-41c1-45c8-80ce-a96f3cda7ae6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:02.450005 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:02.458537 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86751f43-06d3-404c-bc19-47eb9daa8521 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:02.460522 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5060674-41c1-45c8-80ce-a96f3cda7ae6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:22:02.468769 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-d5060674-41c1-45c8-80ce-a96f3cda7ae6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 15:22:02.468769 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:22:02.468769 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:22:02.468769 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:22:02.468769 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:22:02.468769 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:22:02.468769 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:22:02.468769 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:22:02.468769 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:22:02.468769 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:22:02.468769 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:22:02.468769 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:22:02.468769 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:22:02.468769 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:22:02.468769 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:22:02.468769 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:22:02.469627 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:22:02.469627 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:22:02.469627 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:22:02.469627 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:22:02.469627 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:22:02.469627 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:22:02.469627 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:22:02.469627 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:22:02.469627 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:22:02.469627 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:22:02.469627 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:22:02.469627 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:22:02.469627 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:22:02.469627 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:22:02.469627 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:22:02.469627 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:22:02.470381 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:22:02.470381 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:22:02.470381 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:22:02.470381 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:22:02.470381 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:22:02.470381 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:22:02.470381 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:22:02.470381 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:22:02.470381 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:22:02.470381 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:22:02.470381 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 15:22:02.470381 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:22:02.470381 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 36/73] 10.222.0.99 () {56 vars in 1114 bytes} [Thu Jul 14 15:22:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:22:02.471834 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9428dd5f-40f4-4331-8b36-0db0f92d7b7f None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:02.471834 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9428dd5f-40f4-4331-8b36-0db0f92d7b7f None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:02.471834 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9428dd5f-40f4-4331-8b36-0db0f92d7b7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:02.489054 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-9428dd5f-40f4-4331-8b36-0db0f92d7b7f None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:22:02.490387 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-9428dd5f-40f4-4331-8b36-0db0f92d7b7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:22:02.507710 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88df76f5-92f1-4fb3-8940-799044211189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:02.508478 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88df76f5-92f1-4fb3-8940-799044211189 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:02.508798 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88df76f5-92f1-4fb3-8940-799044211189 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:02.509317 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88df76f5-92f1-4fb3-8940-799044211189 None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:02.510508 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88df76f5-92f1-4fb3-8940-799044211189 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:22:02.510828 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88df76f5-92f1-4fb3-8940-799044211189 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:02.512261 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88df76f5-92f1-4fb3-8940-799044211189 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:02.529514 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 38/74] 10.222.0.99 () {56 vars in 1099 bytes} [Thu Jul 14 15:22:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:02.529514 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88df76f5-92f1-4fb3-8940-799044211189 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:02.566708 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 37/75] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:22:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:22:02.567580 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9428dd5f-40f4-4331-8b36-0db0f92d7b7f None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:02.642323 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88df76f5-92f1-4fb3-8940-799044211189 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:02.668424 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9428dd5f-40f4-4331-8b36-0db0f92d7b7f None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:02.669956 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:02.676502 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6712430a-160c-40cb-8b8a-b84d9d4ca3cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:02.677165 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6712430a-160c-40cb-8b8a-b84d9d4ca3cf None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:02.677461 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6712430a-160c-40cb-8b8a-b84d9d4ca3cf None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:02.677746 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6712430a-160c-40cb-8b8a-b84d9d4ca3cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:02.703857 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb04a970-0e28-42c2-bd15-337244d88367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:02.704530 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb04a970-0e28-42c2-bd15-337244d88367 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:02.704818 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb04a970-0e28-42c2-bd15-337244d88367 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:02.705101 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb04a970-0e28-42c2-bd15-337244d88367 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:02.705662 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:02.705866 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb04a970-0e28-42c2-bd15-337244d88367 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:22:02.706168 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb04a970-0e28-42c2-bd15-337244d88367 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:02.707703 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb04a970-0e28-42c2-bd15-337244d88367 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:02.716397 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 38/76] 10.222.0.99 () {56 vars in 1111 bytes} [Thu Jul 14 15:22:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:02.717204 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb04a970-0e28-42c2-bd15-337244d88367 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:02.748138 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6712430a-160c-40cb-8b8a-b84d9d4ca3cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:02.749882 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6712430a-160c-40cb-8b8a-b84d9d4ca3cf None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:02.830914 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb04a970-0e28-42c2-bd15-337244d88367 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:02.855411 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 39/77] 10.222.0.99 () {58 vars in 1282 bytes} [Thu Jul 14 15:22:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 15:22:02.856605 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7f83af-ebb5-4fbe-aed3-3f8a74903588 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:02.856907 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7f83af-ebb5-4fbe-aed3-3f8a74903588 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:02.857197 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7f83af-ebb5-4fbe-aed3-3f8a74903588 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:02.858500 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 40/78] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:22:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:22:02.860857 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7f83af-ebb5-4fbe-aed3-3f8a74903588 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:02.872797 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ac93ff-4045-47ad-8638-026fe3da7f8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:02.873946 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ac93ff-4045-47ad-8638-026fe3da7f8e None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:02.874236 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ac93ff-4045-47ad-8638-026fe3da7f8e None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:02.874519 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ac93ff-4045-47ad-8638-026fe3da7f8e None admin] PATH_INFO: `/v3/users/3e33ee1d9fa74c5fa2f58e221ad2d0a8` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:02.875776 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ac93ff-4045-47ad-8638-026fe3da7f8e None admin] RBAC: Authorizing `identity:get_user(user_id=3e33ee1d9fa74c5fa2f58e221ad2d0a8)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:02.877231 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ac93ff-4045-47ad-8638-026fe3da7f8e None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:02.878872 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 39/79] 10.222.0.99 () {56 vars in 1177 bytes} [Thu Jul 14 15:22:02 2022] GET /identity/v3/users/3e33ee1d9fa74c5fa2f58e221ad2d0a8 => generated 247 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:02.879670 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ac93ff-4045-47ad-8638-026fe3da7f8e None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:02.912590 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:02.923238 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:02.967827 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7f83af-ebb5-4fbe-aed3-3f8a74903588 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:02.982791 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ac93ff-4045-47ad-8638-026fe3da7f8e None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:03.006723 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a49a22-d6a5-493f-84ec-ad624696d6d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:03.008092 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a49a22-d6a5-493f-84ec-ad624696d6d0 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:03.008390 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a49a22-d6a5-493f-84ec-ad624696d6d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:03.008677 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a49a22-d6a5-493f-84ec-ad624696d6d0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:03.018942 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a49a22-d6a5-493f-84ec-ad624696d6d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:22:03.019392 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd0357b-4403-46b5-94d4-4e7619bcc64c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:03.020301 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd0357b-4403-46b5-94d4-4e7619bcc64c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:03.020617 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd0357b-4403-46b5-94d4-4e7619bcc64c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:03.020897 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd0357b-4403-46b5-94d4-4e7619bcc64c None admin] PATH_INFO: `/v3/users/7f5177fcb1524b918c6662c934105643` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:03.022125 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd0357b-4403-46b5-94d4-4e7619bcc64c None admin] RBAC: Authorizing `identity:get_user(user_id=7f5177fcb1524b918c6662c934105643)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:03.023924 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd0357b-4403-46b5-94d4-4e7619bcc64c None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:03.025567 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 40/80] 10.222.0.99 () {56 vars in 1177 bytes} [Thu Jul 14 15:22:02 2022] GET /identity/v3/users/7f5177fcb1524b918c6662c934105643 => generated 275 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:03.026332 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd0357b-4403-46b5-94d4-4e7619bcc64c None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:03.026736 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-82a49a22-d6a5-493f-84ec-ad624696d6d0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 15:22:03.026736 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:22:03.026736 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:22:03.026736 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:22:03.026736 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:22:03.026736 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:22:03.026736 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:22:03.026736 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:22:03.026736 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:22:03.026736 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:22:03.026736 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:22:03.026736 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:22:03.026736 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:22:03.026736 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:22:03.026736 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:22:03.026736 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:22:03.027570 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:22:03.027570 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:22:03.027570 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:22:03.027570 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:22:03.027570 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:22:03.027570 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:22:03.027570 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:22:03.027570 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:22:03.027570 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:22:03.027570 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:22:03.027570 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:22:03.027570 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:22:03.027570 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:22:03.027570 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:22:03.027570 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:22:03.027570 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:22:03.028355 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:22:03.028355 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:22:03.028355 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:22:03.028355 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:22:03.028355 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:22:03.028355 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:22:03.028355 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:22:03.028355 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:22:03.028355 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:22:03.028355 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:22:03.028355 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 15:22:03.028355 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:22:03.028355 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 41/81] 10.222.0.99 () {56 vars in 1099 bytes} [Thu Jul 14 15:22:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:22:03.029420 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de92884-1a41-4d0c-8a02-abefd27c08d5 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:03.029511 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de92884-1a41-4d0c-8a02-abefd27c08d5 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:03.029733 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de92884-1a41-4d0c-8a02-abefd27c08d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:03.046747 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-1de92884-1a41-4d0c-8a02-abefd27c08d5 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:22:03.048006 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-1de92884-1a41-4d0c-8a02-abefd27c08d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:22:03.115245 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 42/82] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:22:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:22:03.116659 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de92884-1a41-4d0c-8a02-abefd27c08d5 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:03.154462 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:03.160013 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd0357b-4403-46b5-94d4-4e7619bcc64c None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:03.173543 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:03.193047 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf08240f-42c6-454c-9c9f-6b4fc56e0b68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:03.193920 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf08240f-42c6-454c-9c9f-6b4fc56e0b68 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:03.194232 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf08240f-42c6-454c-9c9f-6b4fc56e0b68 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:03.194588 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf08240f-42c6-454c-9c9f-6b4fc56e0b68 None admin] PATH_INFO: `/v3/projects/1d9e82a46b1c4b25939e533c26cb44df` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:03.195810 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf08240f-42c6-454c-9c9f-6b4fc56e0b68 None admin] RBAC: Authorizing `identity:get_project(project_id=1d9e82a46b1c4b25939e533c26cb44df)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:03.197263 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf08240f-42c6-454c-9c9f-6b4fc56e0b68 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:03.198957 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 41/83] 10.222.0.99 () {56 vars in 1186 bytes} [Thu Jul 14 15:22:03 2022] GET /identity/v3/projects/1d9e82a46b1c4b25939e533c26cb44df => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:03.199754 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf08240f-42c6-454c-9c9f-6b4fc56e0b68 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:03.232625 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de92884-1a41-4d0c-8a02-abefd27c08d5 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:03.267529 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1684483e-885a-412c-a89a-904d4ed27298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:03.268220 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1684483e-885a-412c-a89a-904d4ed27298 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:03.268521 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1684483e-885a-412c-a89a-904d4ed27298 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:03.268804 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1684483e-885a-412c-a89a-904d4ed27298 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:03.312353 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf08240f-42c6-454c-9c9f-6b4fc56e0b68 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:03.342944 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1684483e-885a-412c-a89a-904d4ed27298 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:03.344464 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1684483e-885a-412c-a89a-904d4ed27298 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:03.347978 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f022970-9331-46b7-9f2c-ec125ebe470a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:03.348634 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f022970-9331-46b7-9f2c-ec125ebe470a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:03.348927 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f022970-9331-46b7-9f2c-ec125ebe470a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:03.349212 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f022970-9331-46b7-9f2c-ec125ebe470a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:03.386129 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:03.393476 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:03.420570 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f022970-9331-46b7-9f2c-ec125ebe470a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:03.422073 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f022970-9331-46b7-9f2c-ec125ebe470a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:03.447016 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 43/84] 10.222.0.99 () {58 vars in 1282 bytes} [Thu Jul 14 15:22:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 15:22:03.447831 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1684483e-885a-412c-a89a-904d4ed27298 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:03.530766 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 42/85] 10.222.0.99 () {58 vars in 1282 bytes} [Thu Jul 14 15:22:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 15:22:03.532234 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f022970-9331-46b7-9f2c-ec125ebe470a None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:03.555139 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1684483e-885a-412c-a89a-904d4ed27298 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:03.590225 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d376f8d5-cd3b-40f2-b383-5abe5e948632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:03.590899 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d376f8d5-cd3b-40f2-b383-5abe5e948632 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:03.591186 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d376f8d5-cd3b-40f2-b383-5abe5e948632 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:03.591475 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d376f8d5-cd3b-40f2-b383-5abe5e948632 None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:03.592264 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d376f8d5-cd3b-40f2-b383-5abe5e948632 None admin] RBAC: Adding query filter params (name=member) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:22:03.592577 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d376f8d5-cd3b-40f2-b383-5abe5e948632 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:03.593969 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d376f8d5-cd3b-40f2-b383-5abe5e948632 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:03.602505 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 44/86] 10.222.0.99 () {56 vars in 1101 bytes} [Thu Jul 14 15:22:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:03.603214 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:03.603362 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d376f8d5-cd3b-40f2-b383-5abe5e948632 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:03.617598 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:03.641957 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f022970-9331-46b7-9f2c-ec125ebe470a None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:03.677726 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef391a4c-7172-474c-99cf-b093ab8097e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:03.678513 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef391a4c-7172-474c-99cf-b093ab8097e2 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:03.679323 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef391a4c-7172-474c-99cf-b093ab8097e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:03.679695 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef391a4c-7172-474c-99cf-b093ab8097e2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:03.680956 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef391a4c-7172-474c-99cf-b093ab8097e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:03.682419 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef391a4c-7172-474c-99cf-b093ab8097e2 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:03.684089 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 43/87] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:22:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:03.684858 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef391a4c-7172-474c-99cf-b093ab8097e2 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:03.708609 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d376f8d5-cd3b-40f2-b383-5abe5e948632 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:03.742652 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfed854-836d-4a4a-88f8-5b6a0ae1f28a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:03.743320 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfed854-836d-4a4a-88f8-5b6a0ae1f28a None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:03.743638 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfed854-836d-4a4a-88f8-5b6a0ae1f28a None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:03.743930 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfed854-836d-4a4a-88f8-5b6a0ae1f28a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:03.744779 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfed854-836d-4a4a-88f8-5b6a0ae1f28a None admin] RBAC: Adding query filter params (role.id=b8c98fddc0fd4436b577fd1d3361bc38, scope.project.id=1d9e82a46b1c4b25939e533c26cb44df, user.id=7f5177fcb1524b918c6662c934105643) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:22:03.745097 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfed854-836d-4a4a-88f8-5b6a0ae1f28a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:03.746639 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfed854-836d-4a4a-88f8-5b6a0ae1f28a None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:03.752391 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 45/88] 10.222.0.99 () {56 vars in 1374 bytes} [Thu Jul 14 15:22:03 2022] GET /identity/v3/role_assignments?user.id=7f5177fcb1524b918c6662c934105643&scope.project.id=1d9e82a46b1c4b25939e533c26cb44df&role.id=b8c98fddc0fd4436b577fd1d3361bc38 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:03.759850 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfed854-836d-4a4a-88f8-5b6a0ae1f28a None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:03.792348 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef391a4c-7172-474c-99cf-b093ab8097e2 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:03.840299 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad17687-5760-407a-bb21-27d3e70263f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:03.841201 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad17687-5760-407a-bb21-27d3e70263f5 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:03.841490 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad17687-5760-407a-bb21-27d3e70263f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:03.841773 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad17687-5760-407a-bb21-27d3e70263f5 None admin] PATH_INFO: `/v3/projects/1d9e82a46b1c4b25939e533c26cb44df` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:03.843354 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad17687-5760-407a-bb21-27d3e70263f5 None admin] RBAC: Authorizing `identity:get_project(project_id=1d9e82a46b1c4b25939e533c26cb44df)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:03.845540 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad17687-5760-407a-bb21-27d3e70263f5 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:03.847687 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 44/89] 10.222.0.99 () {56 vars in 1186 bytes} [Thu Jul 14 15:22:03 2022] GET /identity/v3/projects/1d9e82a46b1c4b25939e533c26cb44df => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:03.848427 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad17687-5760-407a-bb21-27d3e70263f5 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:03.853822 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:03.862260 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:03.896195 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfed854-836d-4a4a-88f8-5b6a0ae1f28a None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:03.935909 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4d8308-405e-45c6-b9f9-f4ec859f076c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:03.936793 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4d8308-405e-45c6-b9f9-f4ec859f076c None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:03.937081 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4d8308-405e-45c6-b9f9-f4ec859f076c None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:03.937362 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4d8308-405e-45c6-b9f9-f4ec859f076c None admin] PATH_INFO: `/v3/projects/1d9e82a46b1c4b25939e533c26cb44df` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:03.939004 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4d8308-405e-45c6-b9f9-f4ec859f076c None admin] RBAC: Authorizing `identity:get_project(project_id=1d9e82a46b1c4b25939e533c26cb44df)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:03.940926 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4d8308-405e-45c6-b9f9-f4ec859f076c None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:03.948672 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 46/90] 10.222.0.99 () {56 vars in 1186 bytes} [Thu Jul 14 15:22:03 2022] GET /identity/v3/projects/1d9e82a46b1c4b25939e533c26cb44df => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:03.948672 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4d8308-405e-45c6-b9f9-f4ec859f076c None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:03.971851 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad17687-5760-407a-bb21-27d3e70263f5 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:04.017651 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76482d6-2caf-4ff8-a665-0a4fadff6981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:04.018537 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76482d6-2caf-4ff8-a665-0a4fadff6981 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:04.018818 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76482d6-2caf-4ff8-a665-0a4fadff6981 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:04.019095 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76482d6-2caf-4ff8-a665-0a4fadff6981 None admin] PATH_INFO: `/v3/users/7f5177fcb1524b918c6662c934105643` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:04.020401 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76482d6-2caf-4ff8-a665-0a4fadff6981 None admin] RBAC: Authorizing `identity:get_user(user_id=7f5177fcb1524b918c6662c934105643)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:04.021843 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76482d6-2caf-4ff8-a665-0a4fadff6981 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:04.024513 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 45/91] 10.222.0.99 () {56 vars in 1177 bytes} [Thu Jul 14 15:22:03 2022] GET /identity/v3/users/7f5177fcb1524b918c6662c934105643 => generated 275 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:04.029446 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76482d6-2caf-4ff8-a665-0a4fadff6981 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:04.061066 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4d8308-405e-45c6-b9f9-f4ec859f076c None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:04.100112 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036d0248-61c9-4267-a835-7b561c32e23a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:04.100841 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036d0248-61c9-4267-a835-7b561c32e23a None admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:04.101133 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036d0248-61c9-4267-a835-7b561c32e23a None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:04.101506 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036d0248-61c9-4267-a835-7b561c32e23a None admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:04.102233 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036d0248-61c9-4267-a835-7b561c32e23a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:04.103714 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036d0248-61c9-4267-a835-7b561c32e23a None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:04.103985 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:04.125536 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:04.131387 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 47/92] 10.222.0.99 () {60 vars in 1132 bytes} [Thu Jul 14 15:22:03 2022] POST /identity/v3/users => generated 289 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:22:04.133062 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036d0248-61c9-4267-a835-7b561c32e23a None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:04.148397 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76482d6-2caf-4ff8-a665-0a4fadff6981 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:04.191279 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6067bf-57ef-4e22-961a-d90e9eb638f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:04.192178 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6067bf-57ef-4e22-961a-d90e9eb638f0 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:04.192564 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6067bf-57ef-4e22-961a-d90e9eb638f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:04.193056 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6067bf-57ef-4e22-961a-d90e9eb638f0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:04.194070 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6067bf-57ef-4e22-961a-d90e9eb638f0 None admin] RBAC: Adding query filter params (role.id=d0f6ba1314f14d9f9c816bdb5d0c3aac, scope.project.id=1d9e82a46b1c4b25939e533c26cb44df, user.id=3e33ee1d9fa74c5fa2f58e221ad2d0a8) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:22:04.195074 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6067bf-57ef-4e22-961a-d90e9eb638f0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:04.196732 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6067bf-57ef-4e22-961a-d90e9eb638f0 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:04.203838 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 46/93] 10.222.0.99 () {56 vars in 1374 bytes} [Thu Jul 14 15:22:04 2022] GET /identity/v3/role_assignments?user.id=3e33ee1d9fa74c5fa2f58e221ad2d0a8&scope.project.id=1d9e82a46b1c4b25939e533c26cb44df&role.id=d0f6ba1314f14d9f9c816bdb5d0c3aac => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:04.204920 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6067bf-57ef-4e22-961a-d90e9eb638f0 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:04.266113 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036d0248-61c9-4267-a835-7b561c32e23a None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:04.309382 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b238b27f-6cb8-4899-a094-36557755889c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:04.310356 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b238b27f-6cb8-4899-a094-36557755889c None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:04.310838 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b238b27f-6cb8-4899-a094-36557755889c None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:04.311276 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b238b27f-6cb8-4899-a094-36557755889c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:04.312422 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b238b27f-6cb8-4899-a094-36557755889c None admin] RBAC: Adding query filter params (role.id=3627b4f45c8c4f41a7def6b15f1b8225, scope.project.id=1d9e82a46b1c4b25939e533c26cb44df, user.id=7f5177fcb1524b918c6662c934105643) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:22:04.312995 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b238b27f-6cb8-4899-a094-36557755889c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:04.314764 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b238b27f-6cb8-4899-a094-36557755889c None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:04.322249 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 48/94] 10.222.0.99 () {56 vars in 1374 bytes} [Thu Jul 14 15:22:04 2022] GET /identity/v3/role_assignments?user.id=7f5177fcb1524b918c6662c934105643&scope.project.id=1d9e82a46b1c4b25939e533c26cb44df&role.id=3627b4f45c8c4f41a7def6b15f1b8225 => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:04.348823 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6067bf-57ef-4e22-961a-d90e9eb638f0 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:04.390048 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6652379e-0804-4653-b4be-79d8818b3d8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:04.390726 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6652379e-0804-4653-b4be-79d8818b3d8c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:04.391019 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6652379e-0804-4653-b4be-79d8818b3d8c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:04.391299 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6652379e-0804-4653-b4be-79d8818b3d8c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:04.396189 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:04.477817 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6652379e-0804-4653-b4be-79d8818b3d8c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:04.479399 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6652379e-0804-4653-b4be-79d8818b3d8c None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:04.601332 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 47/95] 10.222.0.99 () {58 vars in 1282 bytes} [Thu Jul 14 15:22:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 15:22:04.606713 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b238b27f-6cb8-4899-a094-36557755889c None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:04.689275 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:04.733544 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b238b27f-6cb8-4899-a094-36557755889c None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:04.771570 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c85e61-b165-47b7-b7ab-4cdd3bf13ecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:04.772477 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c85e61-b165-47b7-b7ab-4cdd3bf13ecd None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:04.772797 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c85e61-b165-47b7-b7ab-4cdd3bf13ecd None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:04.773081 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c85e61-b165-47b7-b7ab-4cdd3bf13ecd None admin] PATH_INFO: `/v3/projects/f966b0d68ba448648d66bce40b1605eb` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:04.774415 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c85e61-b165-47b7-b7ab-4cdd3bf13ecd None admin] RBAC: Authorizing `identity:get_project(project_id=f966b0d68ba448648d66bce40b1605eb)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:04.775915 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c85e61-b165-47b7-b7ab-4cdd3bf13ecd None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:04.777929 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 49/96] 10.222.0.99 () {56 vars in 1186 bytes} [Thu Jul 14 15:22:04 2022] GET /identity/v3/projects/f966b0d68ba448648d66bce40b1605eb => generated 312 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:04.783730 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6652379e-0804-4653-b4be-79d8818b3d8c None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:04.842778 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:04.923724 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6652379e-0804-4653-b4be-79d8818b3d8c None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:04.962781 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56949764-725e-4fc0-a6d9-2fd87eb52940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:04.963463 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56949764-725e-4fc0-a6d9-2fd87eb52940 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:04.963782 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56949764-725e-4fc0-a6d9-2fd87eb52940 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:04.964068 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56949764-725e-4fc0-a6d9-2fd87eb52940 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:04.964925 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56949764-725e-4fc0-a6d9-2fd87eb52940 None admin] RBAC: Adding query filter params (role.id=b8c98fddc0fd4436b577fd1d3361bc38, scope.project.id=f966b0d68ba448648d66bce40b1605eb, user.id=7f5177fcb1524b918c6662c934105643) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:22:04.965244 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56949764-725e-4fc0-a6d9-2fd87eb52940 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:04.966790 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56949764-725e-4fc0-a6d9-2fd87eb52940 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:04.973050 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 48/97] 10.222.0.99 () {56 vars in 1374 bytes} [Thu Jul 14 15:22:04 2022] GET /identity/v3/role_assignments?user.id=7f5177fcb1524b918c6662c934105643&scope.project.id=f966b0d68ba448648d66bce40b1605eb&role.id=b8c98fddc0fd4436b577fd1d3361bc38 => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:06.544416 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e9aa18-ad47-4e22-9c3d-b26ba73a0dcb None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:06.545030 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e9aa18-ad47-4e22-9c3d-b26ba73a0dcb None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:06.545030 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e9aa18-ad47-4e22-9c3d-b26ba73a0dcb None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:06.546487 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 50/98] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:22:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:22:06.561694 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a7c07b-d085-4e09-b46b-4a8f9c13007e None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:06.562213 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a7c07b-d085-4e09-b46b-4a8f9c13007e None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:06.562305 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a7c07b-d085-4e09-b46b-4a8f9c13007e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:06.584881 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-c0a7c07b-d085-4e09-b46b-4a8f9c13007e None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:22:06.586970 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c0a7c07b-d085-4e09-b46b-4a8f9c13007e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:22:06.623181 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cd8c4b-0144-4ed8-9389-cf74a0368c5b None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:06.623687 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cd8c4b-0144-4ed8-9389-cf74a0368c5b None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:06.623991 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cd8c4b-0144-4ed8-9389-cf74a0368c5b None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:06.625920 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 51/99] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:22:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 15:22:06.637487 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb7b9cd-6d07-4fc3-80f8-21476ee1553c None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:06.637815 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb7b9cd-6d07-4fc3-80f8-21476ee1553c None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:06.638056 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb7b9cd-6d07-4fc3-80f8-21476ee1553c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:06.661939 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-2cb7b9cd-6d07-4fc3-80f8-21476ee1553c None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:22:06.663164 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-2cb7b9cd-6d07-4fc3-80f8-21476ee1553c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:22:06.709347 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 49/100] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:22:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:22:06.749210 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a7c07b-d085-4e09-b46b-4a8f9c13007e None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:06.776945 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 52/101] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:22:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 15:22:06.800067 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:06.837977 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb7b9cd-6d07-4fc3-80f8-21476ee1553c None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:06.916169 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:06.924275 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a7c07b-d085-4e09-b46b-4a8f9c13007e None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:06.958091 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb7b9cd-6d07-4fc3-80f8-21476ee1553c None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:06.982242 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5b1e73-3e3d-4e30-9fa5-754ac5652218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:06.983389 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5b1e73-3e3d-4e30-9fa5-754ac5652218 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:06.983914 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5b1e73-3e3d-4e30-9fa5-754ac5652218 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:06.984455 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5b1e73-3e3d-4e30-9fa5-754ac5652218 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:07.009556 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5b1e73-3e3d-4e30-9fa5-754ac5652218 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:22:07.013523 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec12456-589e-4188-b932-cd6ae3bc2a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:07.014351 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec12456-589e-4188-b932-cd6ae3bc2a2b None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:07.014643 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec12456-589e-4188-b932-cd6ae3bc2a2b None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:07.014924 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec12456-589e-4188-b932-cd6ae3bc2a2b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:07.018907 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-ae5b1e73-3e3d-4e30-9fa5-754ac5652218 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 15:22:07.018907 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:22:07.018907 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:22:07.018907 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:22:07.018907 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:22:07.018907 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:22:07.018907 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:22:07.018907 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:22:07.018907 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:22:07.018907 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:22:07.018907 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:22:07.018907 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:22:07.018907 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:22:07.018907 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:22:07.018907 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:22:07.018907 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:22:07.019816 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:22:07.019816 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:22:07.019816 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:22:07.019816 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:22:07.019816 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:22:07.019816 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:22:07.019816 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:22:07.019816 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:22:07.019816 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:22:07.019816 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:22:07.019816 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:22:07.019816 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:22:07.019816 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:22:07.019816 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:22:07.019816 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:22:07.019816 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:22:07.020611 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:22:07.020611 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:22:07.020611 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:22:07.020611 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:22:07.020611 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:22:07.020611 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:22:07.020611 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:22:07.020611 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:22:07.020611 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:22:07.020611 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:22:07.020611 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 15:22:07.020611 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:22:07.027962 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 50/102] 10.222.0.99 () {56 vars in 1099 bytes} [Thu Jul 14 15:22:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 15:22:07.030772 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5b1e73-3e3d-4e30-9fa5-754ac5652218 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:07.047023 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec12456-589e-4188-b932-cd6ae3bc2a2b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:22:07.055191 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-7ec12456-589e-4188-b932-cd6ae3bc2a2b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:22:07.055191 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:22:07.055191 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:22:07.055191 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:22:07.055191 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:22:07.055191 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:22:07.055191 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:22:07.055191 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:22:07.055191 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:22:07.055191 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:22:07.055191 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:22:07.055191 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:22:07.055191 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:22:07.055191 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:22:07.055191 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:22:07.055191 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:22:07.056080 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:22:07.056080 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:22:07.056080 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:22:07.056080 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:22:07.056080 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:22:07.056080 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:22:07.056080 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:22:07.056080 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:22:07.056080 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:22:07.056080 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:22:07.056080 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:22:07.056080 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:22:07.056080 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:22:07.056080 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:22:07.056080 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:22:07.056080 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:22:07.056840 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:22:07.056840 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:22:07.056840 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:22:07.056840 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:22:07.056840 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:22:07.056840 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:22:07.056840 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:22:07.056840 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:22:07.056840 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:22:07.056840 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:22:07.056840 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:22:07.056840 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:22:07.064499 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 53/103] 10.222.0.99 () {56 vars in 1096 bytes} [Thu Jul 14 15:22:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 15:22:07.072596 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec12456-589e-4188-b932-cd6ae3bc2a2b None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:07.141362 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:07.154566 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5b1e73-3e3d-4e30-9fa5-754ac5652218 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:07.193401 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0419de-1983-486d-83bc-b2b9cf7f1735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:07.194443 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0419de-1983-486d-83bc-b2b9cf7f1735 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:07.194987 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0419de-1983-486d-83bc-b2b9cf7f1735 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:07.196101 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0419de-1983-486d-83bc-b2b9cf7f1735 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:07.197103 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0419de-1983-486d-83bc-b2b9cf7f1735 None admin] RBAC: Adding query filter params (name=member) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:22:07.197642 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0419de-1983-486d-83bc-b2b9cf7f1735 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:07.199301 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0419de-1983-486d-83bc-b2b9cf7f1735 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:07.214532 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 51/104] 10.222.0.99 () {56 vars in 1101 bytes} [Thu Jul 14 15:22:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:07.216505 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060082fe-db63-4825-86b1-9bcb737e86b5 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:07.217024 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060082fe-db63-4825-86b1-9bcb737e86b5 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:07.217515 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060082fe-db63-4825-86b1-9bcb737e86b5 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:07.219510 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 52/105] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:22:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:22:07.220841 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060082fe-db63-4825-86b1-9bcb737e86b5 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:07.228784 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec12456-589e-4188-b932-cd6ae3bc2a2b None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:07.248779 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:07.272890 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c0232a-6d43-4513-8012-06ee45542102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:07.275193 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c0232a-6d43-4513-8012-06ee45542102 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:07.275486 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c0232a-6d43-4513-8012-06ee45542102 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:07.275788 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c0232a-6d43-4513-8012-06ee45542102 None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:07.276556 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c0232a-6d43-4513-8012-06ee45542102 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:22:07.276869 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c0232a-6d43-4513-8012-06ee45542102 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:07.282981 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c0232a-6d43-4513-8012-06ee45542102 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:07.306239 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 54/106] 10.222.0.99 () {56 vars in 1099 bytes} [Thu Jul 14 15:22:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:07.314385 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007a567d-cfb9-4e28-93d6-8fc80ff718e9 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:07.314797 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007a567d-cfb9-4e28-93d6-8fc80ff718e9 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:07.315097 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007a567d-cfb9-4e28-93d6-8fc80ff718e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:07.354049 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-007a567d-cfb9-4e28-93d6-8fc80ff718e9 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:22:07.355276 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-007a567d-cfb9-4e28-93d6-8fc80ff718e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:22:07.424043 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060082fe-db63-4825-86b1-9bcb737e86b5 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:07.464377 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbd2183-51e2-4454-b600-ee9e08fec135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:07.465272 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbd2183-51e2-4454-b600-ee9e08fec135 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:07.465989 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbd2183-51e2-4454-b600-ee9e08fec135 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:07.466283 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbd2183-51e2-4454-b600-ee9e08fec135 None admin] PATH_INFO: `/v3/users/7f5177fcb1524b918c6662c934105643` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:07.467511 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbd2183-51e2-4454-b600-ee9e08fec135 None admin] RBAC: Authorizing `identity:get_user(user_id=7f5177fcb1524b918c6662c934105643)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:07.468987 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbd2183-51e2-4454-b600-ee9e08fec135 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:07.470948 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 53/107] 10.222.0.99 () {56 vars in 1177 bytes} [Thu Jul 14 15:22:07 2022] GET /identity/v3/users/7f5177fcb1524b918c6662c934105643 => generated 275 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:07.471519 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbd2183-51e2-4454-b600-ee9e08fec135 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:07.497575 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:07.573417 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 55/108] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:22:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:22:07.574243 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007a567d-cfb9-4e28-93d6-8fc80ff718e9 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:07.611719 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbd2183-51e2-4454-b600-ee9e08fec135 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:07.659224 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f673159-55c0-4d61-bbd6-5a00ab354edf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:07.660148 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f673159-55c0-4d61-bbd6-5a00ab354edf None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:07.660440 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f673159-55c0-4d61-bbd6-5a00ab354edf None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:07.660774 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f673159-55c0-4d61-bbd6-5a00ab354edf None admin] PATH_INFO: `/v3/users/3e33ee1d9fa74c5fa2f58e221ad2d0a8` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:07.662036 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f673159-55c0-4d61-bbd6-5a00ab354edf None admin] RBAC: Authorizing `identity:get_user(user_id=3e33ee1d9fa74c5fa2f58e221ad2d0a8)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:07.663559 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f673159-55c0-4d61-bbd6-5a00ab354edf None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:07.666551 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 54/109] 10.222.0.99 () {56 vars in 1177 bytes} [Thu Jul 14 15:22:07 2022] GET /identity/v3/users/3e33ee1d9fa74c5fa2f58e221ad2d0a8 => generated 247 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:07.667138 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f673159-55c0-4d61-bbd6-5a00ab354edf None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:07.775968 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:07.795088 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f673159-55c0-4d61-bbd6-5a00ab354edf None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:07.836187 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89563edf-f5b1-43ba-aa85-c3d511640623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:07.836987 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89563edf-f5b1-43ba-aa85-c3d511640623 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:07.837283 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89563edf-f5b1-43ba-aa85-c3d511640623 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:07.837570 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89563edf-f5b1-43ba-aa85-c3d511640623 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:07.848903 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89563edf-f5b1-43ba-aa85-c3d511640623 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:22:07.858292 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-89563edf-f5b1-43ba-aa85-c3d511640623 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 15:22:07.858292 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:22:07.858292 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:22:07.858292 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:22:07.858292 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:22:07.858292 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:22:07.858292 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:22:07.858292 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:22:07.858292 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:22:07.858292 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:22:07.858292 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:22:07.858292 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:22:07.858292 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:22:07.858292 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:22:07.858292 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:22:07.858292 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:22:07.859178 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:22:07.859178 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:22:07.859178 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:22:07.859178 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:22:07.859178 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:22:07.859178 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:22:07.859178 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:22:07.859178 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:22:07.859178 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:22:07.859178 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:22:07.859178 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:22:07.859178 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:22:07.859178 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:22:07.859178 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:22:07.859178 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:22:07.859178 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:22:07.859966 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:22:07.859966 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:22:07.859966 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:22:07.859966 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:22:07.859966 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:22:07.859966 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:22:07.859966 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:22:07.859966 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:22:07.859966 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:22:07.859966 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:22:07.859966 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 15:22:07.859966 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:22:07.859966 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 55/110] 10.222.0.99 () {56 vars in 1114 bytes} [Thu Jul 14 15:22:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:22:07.861371 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89563edf-f5b1-43ba-aa85-c3d511640623 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:07.861485 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007a567d-cfb9-4e28-93d6-8fc80ff718e9 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:07.879356 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:07.907814 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb06506-8ed6-460d-a14e-cc0de70b2469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:07.909537 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb06506-8ed6-460d-a14e-cc0de70b2469 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:07.909970 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb06506-8ed6-460d-a14e-cc0de70b2469 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:07.910410 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb06506-8ed6-460d-a14e-cc0de70b2469 None admin] PATH_INFO: `/v3/projects/1d9e82a46b1c4b25939e533c26cb44df` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:07.912292 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb06506-8ed6-460d-a14e-cc0de70b2469 None admin] RBAC: Authorizing `identity:get_project(project_id=1d9e82a46b1c4b25939e533c26cb44df)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:07.914845 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb06506-8ed6-460d-a14e-cc0de70b2469 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:07.917368 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 56/111] 10.222.0.99 () {56 vars in 1186 bytes} [Thu Jul 14 15:22:07 2022] GET /identity/v3/projects/1d9e82a46b1c4b25939e533c26cb44df => generated 298 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:07.919366 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcfbfde-6279-484d-af52-5461d560b13f None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:07.919914 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcfbfde-6279-484d-af52-5461d560b13f None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:07.920632 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcfbfde-6279-484d-af52-5461d560b13f None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:07.922528 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 57/112] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:22:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:22:07.923517 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcfbfde-6279-484d-af52-5461d560b13f None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:08.003201 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89563edf-f5b1-43ba-aa85-c3d511640623 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:08.041264 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc29616-e9ec-4cfb-85d9-78224bbfee8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:08.042979 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc29616-e9ec-4cfb-85d9-78224bbfee8c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:08.043421 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc29616-e9ec-4cfb-85d9-78224bbfee8c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:08.044002 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc29616-e9ec-4cfb-85d9-78224bbfee8c None admin] PATH_INFO: `/v3/projects/1d9e82a46b1c4b25939e533c26cb44df` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:08.045849 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc29616-e9ec-4cfb-85d9-78224bbfee8c None admin] RBAC: Authorizing `identity:get_project(project_id=1d9e82a46b1c4b25939e533c26cb44df)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:08.047503 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc29616-e9ec-4cfb-85d9-78224bbfee8c None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:08.048612 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcfbfde-6279-484d-af52-5461d560b13f None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:08.053925 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 56/113] 10.222.0.99 () {56 vars in 1186 bytes} [Thu Jul 14 15:22:07 2022] GET /identity/v3/projects/1d9e82a46b1c4b25939e533c26cb44df => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:08.053925 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc29616-e9ec-4cfb-85d9-78224bbfee8c None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:08.079322 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:08.085448 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83db65c8-9d8d-4b0b-a21e-9a59bc488d9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:08.086569 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83db65c8-9d8d-4b0b-a21e-9a59bc488d9e None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:08.086886 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83db65c8-9d8d-4b0b-a21e-9a59bc488d9e None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:08.087246 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83db65c8-9d8d-4b0b-a21e-9a59bc488d9e None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:08.088052 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83db65c8-9d8d-4b0b-a21e-9a59bc488d9e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:22:08.088372 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83db65c8-9d8d-4b0b-a21e-9a59bc488d9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:08.089801 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83db65c8-9d8d-4b0b-a21e-9a59bc488d9e None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:08.098714 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 58/114] 10.222.0.99 () {56 vars in 1111 bytes} [Thu Jul 14 15:22:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:08.102500 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7d6666-d9e5-49da-80d2-77a08c125ddc None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:08.103313 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7d6666-d9e5-49da-80d2-77a08c125ddc None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:08.103649 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7d6666-d9e5-49da-80d2-77a08c125ddc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:08.123376 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-6b7d6666-d9e5-49da-80d2-77a08c125ddc None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:22:08.125007 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-6b7d6666-d9e5-49da-80d2-77a08c125ddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:22:08.165917 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:08.172957 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc29616-e9ec-4cfb-85d9-78224bbfee8c None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:08.202710 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 59/115] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:22:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:22:08.204018 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3552255-91b5-4e6b-bbc1-08ff07792ff0 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:08.204390 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3552255-91b5-4e6b-bbc1-08ff07792ff0 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:08.204751 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3552255-91b5-4e6b-bbc1-08ff07792ff0 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:08.206240 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 60/116] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:22:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:22:08.210972 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3552255-91b5-4e6b-bbc1-08ff07792ff0 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:08.212502 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74766f19-2403-437d-beb0-72d954385d41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:08.213933 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74766f19-2403-437d-beb0-72d954385d41 None admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:08.214436 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74766f19-2403-437d-beb0-72d954385d41 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:08.215862 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74766f19-2403-437d-beb0-72d954385d41 None admin] PATH_INFO: `/v3/projects/1d9e82a46b1c4b25939e533c26cb44df/users/7f5177fcb1524b918c6662c934105643/roles/b8c98fddc0fd4436b577fd1d3361bc38` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:08.218459 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74766f19-2403-437d-beb0-72d954385d41 None admin] RBAC: Authorizing `identity:create_grant(project_id=1d9e82a46b1c4b25939e533c26cb44df, user_id=7f5177fcb1524b918c6662c934105643, role_id=b8c98fddc0fd4436b577fd1d3361bc38)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:08.220761 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74766f19-2403-437d-beb0-72d954385d41 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:08.241065 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 57/117] 10.222.0.99 () {58 vars in 1439 bytes} [Thu Jul 14 15:22:08 2022] PUT /identity/v3/projects/1d9e82a46b1c4b25939e533c26cb44df/users/7f5177fcb1524b918c6662c934105643/roles/b8c98fddc0fd4436b577fd1d3361bc38 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:22:08.243280 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74766f19-2403-437d-beb0-72d954385d41 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:08.349407 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3552255-91b5-4e6b-bbc1-08ff07792ff0 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:08.349615 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:08.376397 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74766f19-2403-437d-beb0-72d954385d41 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:08.394782 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b1f8bd-7a15-4cd2-ad8f-16c3dc232a5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:08.395739 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b1f8bd-7a15-4cd2-ad8f-16c3dc232a5a None admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:08.396032 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b1f8bd-7a15-4cd2-ad8f-16c3dc232a5a None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:08.396316 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b1f8bd-7a15-4cd2-ad8f-16c3dc232a5a None admin] PATH_INFO: `/v3/projects/1d9e82a46b1c4b25939e533c26cb44df/users/3e33ee1d9fa74c5fa2f58e221ad2d0a8/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:08.398964 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b1f8bd-7a15-4cd2-ad8f-16c3dc232a5a None admin] RBAC: Authorizing `identity:create_grant(project_id=1d9e82a46b1c4b25939e533c26cb44df, user_id=3e33ee1d9fa74c5fa2f58e221ad2d0a8, role_id=d0f6ba1314f14d9f9c816bdb5d0c3aac)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:08.400648 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b1f8bd-7a15-4cd2-ad8f-16c3dc232a5a None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:08.418345 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 61/118] 10.222.0.99 () {58 vars in 1439 bytes} [Thu Jul 14 15:22:08 2022] PUT /identity/v3/projects/1d9e82a46b1c4b25939e533c26cb44df/users/3e33ee1d9fa74c5fa2f58e221ad2d0a8/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:22:08.419642 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8e9013-93c8-40d6-934b-468e26475846 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:08.421916 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8e9013-93c8-40d6-934b-468e26475846 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:08.421916 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8e9013-93c8-40d6-934b-468e26475846 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:08.425461 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2bdfdb-29fe-4dc7-ad36-c1c8e5e89319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:08.426674 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2bdfdb-29fe-4dc7-ad36-c1c8e5e89319 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:08.426965 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2bdfdb-29fe-4dc7-ad36-c1c8e5e89319 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:08.428590 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2bdfdb-29fe-4dc7-ad36-c1c8e5e89319 None admin] PATH_INFO: `/v3/users/7f5177fcb1524b918c6662c934105643` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:08.431256 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2bdfdb-29fe-4dc7-ad36-c1c8e5e89319 None admin] RBAC: Authorizing `identity:get_user(user_id=7f5177fcb1524b918c6662c934105643)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:08.432994 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2bdfdb-29fe-4dc7-ad36-c1c8e5e89319 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:08.435066 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 58/119] 10.222.0.99 () {56 vars in 1177 bytes} [Thu Jul 14 15:22:08 2022] GET /identity/v3/users/7f5177fcb1524b918c6662c934105643 => generated 275 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:08.436630 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2bdfdb-29fe-4dc7-ad36-c1c8e5e89319 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:08.447908 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-ab8e9013-93c8-40d6-934b-468e26475846 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:22:08.449186 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-ab8e9013-93c8-40d6-934b-468e26475846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:22:08.486572 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:08.552377 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 62/120] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:22:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:22:08.552377 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522ed861-f84f-4cc5-a4fa-3c9e4b1d3f1e None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:08.552377 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522ed861-f84f-4cc5-a4fa-3c9e4b1d3f1e None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:08.552377 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522ed861-f84f-4cc5-a4fa-3c9e4b1d3f1e None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:08.552756 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 63/121] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:22:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:22:08.562533 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522ed861-f84f-4cc5-a4fa-3c9e4b1d3f1e None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:08.602997 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2bdfdb-29fe-4dc7-ad36-c1c8e5e89319 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:08.644332 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4ff322-3590-42a7-8ebe-6ed4ef5b8d24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:08.645134 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4ff322-3590-42a7-8ebe-6ed4ef5b8d24 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:08.645419 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4ff322-3590-42a7-8ebe-6ed4ef5b8d24 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:08.645701 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4ff322-3590-42a7-8ebe-6ed4ef5b8d24 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:08.646882 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4ff322-3590-42a7-8ebe-6ed4ef5b8d24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:08.649570 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4ff322-3590-42a7-8ebe-6ed4ef5b8d24 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:08.650951 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 59/122] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:22:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:08.656119 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c891f191-2a57-465c-9e12-8654bac625b1 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:08.656449 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c891f191-2a57-465c-9e12-8654bac625b1 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:08.656836 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c891f191-2a57-465c-9e12-8654bac625b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:08.678240 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-c891f191-2a57-465c-9e12-8654bac625b1 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:22:08.680108 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c891f191-2a57-465c-9e12-8654bac625b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:22:08.688533 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:08.702189 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522ed861-f84f-4cc5-a4fa-3c9e4b1d3f1e None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:08.739772 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d8e850-4573-4b6f-beaa-66f26265718e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:08.740775 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d8e850-4573-4b6f-beaa-66f26265718e None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:08.741076 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d8e850-4573-4b6f-beaa-66f26265718e None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:08.741367 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d8e850-4573-4b6f-beaa-66f26265718e None admin] PATH_INFO: `/v3/projects/1d9e82a46b1c4b25939e533c26cb44df` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:08.743408 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d8e850-4573-4b6f-beaa-66f26265718e None admin] RBAC: Authorizing `identity:get_project(project_id=1d9e82a46b1c4b25939e533c26cb44df)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:08.745162 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d8e850-4573-4b6f-beaa-66f26265718e None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:08.746955 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 64/123] 10.222.0.99 () {56 vars in 1186 bytes} [Thu Jul 14 15:22:08 2022] GET /identity/v3/projects/1d9e82a46b1c4b25939e533c26cb44df => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:08.747969 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d8e850-4573-4b6f-beaa-66f26265718e None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:08.766544 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 60/124] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:22:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:22:08.767558 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c891f191-2a57-465c-9e12-8654bac625b1 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:08.783676 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:08.889151 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d8e850-4573-4b6f-beaa-66f26265718e None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:08.913606 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c891f191-2a57-465c-9e12-8654bac625b1 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:08.927371 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bed7b0-65de-4437-8b2d-3241fdc199c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:08.928512 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bed7b0-65de-4437-8b2d-3241fdc199c1 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:08.928963 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bed7b0-65de-4437-8b2d-3241fdc199c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:08.929389 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bed7b0-65de-4437-8b2d-3241fdc199c1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:08.942034 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bed7b0-65de-4437-8b2d-3241fdc199c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:22:08.950959 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-f3bed7b0-65de-4437-8b2d-3241fdc199c1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 15:22:08.950959 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:22:08.950959 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:22:08.950959 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:22:08.950959 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:22:08.950959 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:22:08.950959 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:22:08.950959 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:22:08.950959 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:22:08.950959 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:22:08.950959 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:22:08.950959 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:22:08.950959 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:22:08.950959 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:22:08.950959 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:22:08.950959 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:22:08.952010 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:22:08.952010 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:22:08.952010 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:22:08.952010 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:22:08.952010 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:22:08.952010 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:22:08.952010 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:22:08.952010 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:22:08.952010 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:22:08.952010 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:22:08.952010 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:22:08.952010 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:22:08.952010 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:22:08.952010 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:22:08.952010 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:22:08.952010 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:22:08.952924 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:22:08.952924 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:22:08.952924 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:22:08.952924 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:22:08.952924 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:22:08.952924 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:22:08.952924 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:22:08.952924 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:22:08.952924 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:22:08.952924 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:22:08.952924 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 15:22:08.952924 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:22:08.955469 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 65/125] 10.222.0.99 () {56 vars in 1099 bytes} [Thu Jul 14 15:22:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:22:08.961146 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bed7b0-65de-4437-8b2d-3241fdc199c1 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:08.963047 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02458027-d499-486f-a8b0-781c917c9e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:08.963762 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02458027-d499-486f-a8b0-781c917c9e80 None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:08.964058 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02458027-d499-486f-a8b0-781c917c9e80 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:08.964344 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02458027-d499-486f-a8b0-781c917c9e80 None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:08.965082 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02458027-d499-486f-a8b0-781c917c9e80 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:08.967966 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02458027-d499-486f-a8b0-781c917c9e80 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:08.985535 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:09.009794 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 61/126] 10.222.0.99 () {60 vars in 1141 bytes} [Thu Jul 14 15:22:08 2022] POST /identity/v3/projects => generated 302 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:22:09.010759 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02458027-d499-486f-a8b0-781c917c9e80 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:09.156071 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02458027-d499-486f-a8b0-781c917c9e80 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:09.161705 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:09.186630 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bed7b0-65de-4437-8b2d-3241fdc199c1 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:09.202372 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbcee2d-6759-4bcb-ae53-df0ba36723e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:09.203204 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbcee2d-6759-4bcb-ae53-df0ba36723e9 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:09.203500 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbcee2d-6759-4bcb-ae53-df0ba36723e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:09.203821 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbcee2d-6759-4bcb-ae53-df0ba36723e9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:09.218504 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbcee2d-6759-4bcb-ae53-df0ba36723e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:22:09.227722 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-ecbcee2d-6759-4bcb-ae53-df0ba36723e9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 15:22:09.227722 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:22:09.227722 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:22:09.227722 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:22:09.227722 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:22:09.227722 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:22:09.227722 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:22:09.227722 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:22:09.227722 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:22:09.227722 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:22:09.227722 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:22:09.227722 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:22:09.227722 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:22:09.227722 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:22:09.227722 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:22:09.227722 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:22:09.228640 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:22:09.228640 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:22:09.228640 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:22:09.228640 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:22:09.228640 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:22:09.228640 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:22:09.228640 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:22:09.228640 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:22:09.228640 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:22:09.228640 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:22:09.228640 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:22:09.228640 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:22:09.228640 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:22:09.228640 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:22:09.228640 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:22:09.228640 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:22:09.229395 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:22:09.229395 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:22:09.229395 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:22:09.229395 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:22:09.229395 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:22:09.229395 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:22:09.229395 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:22:09.229395 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:22:09.229395 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:22:09.229395 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:22:09.229395 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 15:22:09.229395 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:22:09.234026 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 62/127] 10.222.0.99 () {56 vars in 1099 bytes} [Thu Jul 14 15:22:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:22:09.235849 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbcee2d-6759-4bcb-ae53-df0ba36723e9 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:09.287100 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0111cd8-2cb9-4384-b831-ded9ffed3b97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:09.288047 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0111cd8-2cb9-4384-b831-ded9ffed3b97 None admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:09.288339 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0111cd8-2cb9-4384-b831-ded9ffed3b97 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:09.288626 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0111cd8-2cb9-4384-b831-ded9ffed3b97 None admin] PATH_INFO: `/v3/projects/1d9e82a46b1c4b25939e533c26cb44df/users/7f5177fcb1524b918c6662c934105643/roles/3627b4f45c8c4f41a7def6b15f1b8225` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:09.294131 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0111cd8-2cb9-4384-b831-ded9ffed3b97 None admin] RBAC: Authorizing `identity:create_grant(project_id=1d9e82a46b1c4b25939e533c26cb44df, user_id=7f5177fcb1524b918c6662c934105643, role_id=3627b4f45c8c4f41a7def6b15f1b8225)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:09.295862 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0111cd8-2cb9-4384-b831-ded9ffed3b97 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:09.306603 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:09.322866 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 66/128] 10.222.0.99 () {58 vars in 1439 bytes} [Thu Jul 14 15:22:08 2022] PUT /identity/v3/projects/1d9e82a46b1c4b25939e533c26cb44df/users/7f5177fcb1524b918c6662c934105643/roles/3627b4f45c8c4f41a7def6b15f1b8225 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:22:09.324383 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0111cd8-2cb9-4384-b831-ded9ffed3b97 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:09.370814 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbcee2d-6759-4bcb-ae53-df0ba36723e9 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:09.413701 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3e05d9-33af-41aa-ae2d-9e6e8e9295d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:09.414705 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3e05d9-33af-41aa-ae2d-9e6e8e9295d5 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:09.415128 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3e05d9-33af-41aa-ae2d-9e6e8e9295d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:09.415659 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3e05d9-33af-41aa-ae2d-9e6e8e9295d5 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:09.416798 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3e05d9-33af-41aa-ae2d-9e6e8e9295d5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:22:09.417418 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3e05d9-33af-41aa-ae2d-9e6e8e9295d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:09.418892 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3e05d9-33af-41aa-ae2d-9e6e8e9295d5 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:09.429089 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 63/129] 10.222.0.99 () {56 vars in 1101 bytes} [Thu Jul 14 15:22:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:09.441053 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3e05d9-33af-41aa-ae2d-9e6e8e9295d5 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:09.496978 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0111cd8-2cb9-4384-b831-ded9ffed3b97 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:09.559978 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d23727e-17f1-48e7-bd4f-f15bc54f5f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:09.560756 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d23727e-17f1-48e7-bd4f-f15bc54f5f0f None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:09.561050 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d23727e-17f1-48e7-bd4f-f15bc54f5f0f None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:09.561333 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d23727e-17f1-48e7-bd4f-f15bc54f5f0f None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:09.562122 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d23727e-17f1-48e7-bd4f-f15bc54f5f0f None admin] RBAC: Adding query filter params (name=member) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:22:09.563631 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d23727e-17f1-48e7-bd4f-f15bc54f5f0f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:09.564043 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d23727e-17f1-48e7-bd4f-f15bc54f5f0f None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:09.581952 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 67/130] 10.222.0.99 () {56 vars in 1101 bytes} [Thu Jul 14 15:22:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:09.591849 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d23727e-17f1-48e7-bd4f-f15bc54f5f0f None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:09.620460 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:09.666174 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3e05d9-33af-41aa-ae2d-9e6e8e9295d5 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:09.698723 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:09.735224 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811fc12e-11f3-4d33-92d6-14dfc4006f63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:09.736526 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811fc12e-11f3-4d33-92d6-14dfc4006f63 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:09.737020 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811fc12e-11f3-4d33-92d6-14dfc4006f63 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:09.737502 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811fc12e-11f3-4d33-92d6-14dfc4006f63 None admin] PATH_INFO: `/v3/users/c83c274d5da64fed90c95ef77dcaa254` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:09.757128 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d23727e-17f1-48e7-bd4f-f15bc54f5f0f None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:09.759960 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811fc12e-11f3-4d33-92d6-14dfc4006f63 None admin] RBAC: Authorizing `identity:get_user(user_id=c83c274d5da64fed90c95ef77dcaa254)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:09.761679 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811fc12e-11f3-4d33-92d6-14dfc4006f63 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:09.763652 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 64/131] 10.222.0.99 () {56 vars in 1177 bytes} [Thu Jul 14 15:22:09 2022] GET /identity/v3/users/c83c274d5da64fed90c95ef77dcaa254 => generated 289 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:22:09.773254 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811fc12e-11f3-4d33-92d6-14dfc4006f63 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:09.822553 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7235a130-4977-4f0f-bb72-72b3be4660e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:09.823451 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7235a130-4977-4f0f-bb72-72b3be4660e1 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:09.823770 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7235a130-4977-4f0f-bb72-72b3be4660e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:09.824057 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7235a130-4977-4f0f-bb72-72b3be4660e1 None admin] PATH_INFO: `/v3/users/7f5177fcb1524b918c6662c934105643` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:09.825338 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7235a130-4977-4f0f-bb72-72b3be4660e1 None admin] RBAC: Authorizing `identity:get_user(user_id=7f5177fcb1524b918c6662c934105643)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:09.826855 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7235a130-4977-4f0f-bb72-72b3be4660e1 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:09.837251 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 68/132] 10.222.0.99 () {56 vars in 1177 bytes} [Thu Jul 14 15:22:09 2022] GET /identity/v3/users/7f5177fcb1524b918c6662c934105643 => generated 275 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:22:09.843829 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7235a130-4977-4f0f-bb72-72b3be4660e1 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:10.024786 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:10.025546 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811fc12e-11f3-4d33-92d6-14dfc4006f63 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:10.058701 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7235a130-4977-4f0f-bb72-72b3be4660e1 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:10.110320 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11abb577-6637-44fc-8848-4dc51ce4ba59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:10.110978 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11abb577-6637-44fc-8848-4dc51ce4ba59 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:10.111271 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11abb577-6637-44fc-8848-4dc51ce4ba59 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:10.111553 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11abb577-6637-44fc-8848-4dc51ce4ba59 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:10.130359 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c7374b-2a61-4b6c-8958-96c2d6e7c0fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:10.131666 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c7374b-2a61-4b6c-8958-96c2d6e7c0fe None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:10.132163 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c7374b-2a61-4b6c-8958-96c2d6e7c0fe None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:10.132645 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c7374b-2a61-4b6c-8958-96c2d6e7c0fe None admin] PATH_INFO: `/v3/projects/f966b0d68ba448648d66bce40b1605eb` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:10.140975 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c7374b-2a61-4b6c-8958-96c2d6e7c0fe None admin] RBAC: Authorizing `identity:get_project(project_id=f966b0d68ba448648d66bce40b1605eb)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:10.142763 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c7374b-2a61-4b6c-8958-96c2d6e7c0fe None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:10.144806 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 69/133] 10.222.0.99 () {56 vars in 1186 bytes} [Thu Jul 14 15:22:09 2022] GET /identity/v3/projects/f966b0d68ba448648d66bce40b1605eb => generated 312 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:10.151248 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c7374b-2a61-4b6c-8958-96c2d6e7c0fe None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:10.188159 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:10.221827 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11abb577-6637-44fc-8848-4dc51ce4ba59 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:10.223387 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11abb577-6637-44fc-8848-4dc51ce4ba59 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:10.363284 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c7374b-2a61-4b6c-8958-96c2d6e7c0fe None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:10.446125 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0270a924-cd3d-46fc-b460-29a7eb16d37c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:10.447335 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0270a924-cd3d-46fc-b460-29a7eb16d37c None admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:10.448103 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0270a924-cd3d-46fc-b460-29a7eb16d37c None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:10.448718 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0270a924-cd3d-46fc-b460-29a7eb16d37c None admin] PATH_INFO: `/v3/projects/f966b0d68ba448648d66bce40b1605eb/users/7f5177fcb1524b918c6662c934105643/roles/b8c98fddc0fd4436b577fd1d3361bc38` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:10.457317 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0270a924-cd3d-46fc-b460-29a7eb16d37c None admin] RBAC: Authorizing `identity:create_grant(project_id=f966b0d68ba448648d66bce40b1605eb, user_id=7f5177fcb1524b918c6662c934105643, role_id=b8c98fddc0fd4436b577fd1d3361bc38)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:10.459734 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0270a924-cd3d-46fc-b460-29a7eb16d37c None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:10.477985 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 65/134] 10.222.0.99 () {58 vars in 1282 bytes} [Thu Jul 14 15:22:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 706 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 14 15:22:10.484083 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11abb577-6637-44fc-8848-4dc51ce4ba59 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:10.496736 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 70/135] 10.222.0.99 () {58 vars in 1439 bytes} [Thu Jul 14 15:22:10 2022] PUT /identity/v3/projects/f966b0d68ba448648d66bce40b1605eb/users/7f5177fcb1524b918c6662c934105643/roles/b8c98fddc0fd4436b577fd1d3361bc38 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:22:10.598000 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:10.632444 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11abb577-6637-44fc-8848-4dc51ce4ba59 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:10.684773 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a52e7e-8389-4e51-a732-1c77758452d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:10.686332 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a52e7e-8389-4e51-a732-1c77758452d2 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:10.686772 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a52e7e-8389-4e51-a732-1c77758452d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:10.687228 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a52e7e-8389-4e51-a732-1c77758452d2 None admin] PATH_INFO: `/v3/projects/1d9e82a46b1c4b25939e533c26cb44df` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:10.692368 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a52e7e-8389-4e51-a732-1c77758452d2 None admin] RBAC: Authorizing `identity:get_project(project_id=1d9e82a46b1c4b25939e533c26cb44df)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:10.694003 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a52e7e-8389-4e51-a732-1c77758452d2 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:10.695968 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 66/136] 10.222.0.99 () {56 vars in 1186 bytes} [Thu Jul 14 15:22:10 2022] GET /identity/v3/projects/1d9e82a46b1c4b25939e533c26cb44df => generated 298 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:22:10.702163 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0270a924-cd3d-46fc-b460-29a7eb16d37c None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:10.714831 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:10.830151 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0270a924-cd3d-46fc-b460-29a7eb16d37c None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:10.869056 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9197b1-8745-436f-b54e-4108ea617a65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:10.869740 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9197b1-8745-436f-b54e-4108ea617a65 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:10.870036 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9197b1-8745-436f-b54e-4108ea617a65 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:10.870325 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9197b1-8745-436f-b54e-4108ea617a65 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:10.871245 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9197b1-8745-436f-b54e-4108ea617a65 None admin] RBAC: Adding query filter params (role.id=cc5c3091b148423684e5e966150bc6c6, scope.project.id=1d9e82a46b1c4b25939e533c26cb44df, user.id=c83c274d5da64fed90c95ef77dcaa254) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:22:10.871573 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9197b1-8745-436f-b54e-4108ea617a65 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:10.873073 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9197b1-8745-436f-b54e-4108ea617a65 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:10.884260 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 71/137] 10.222.0.99 () {56 vars in 1374 bytes} [Thu Jul 14 15:22:10 2022] GET /identity/v3/role_assignments?user.id=c83c274d5da64fed90c95ef77dcaa254&scope.project.id=1d9e82a46b1c4b25939e533c26cb44df&role.id=cc5c3091b148423684e5e966150bc6c6 => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:10.909568 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf161f5-4a15-4df6-af16-7e049c40696a None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:10.910039 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf161f5-4a15-4df6-af16-7e049c40696a None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:10.910499 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf161f5-4a15-4df6-af16-7e049c40696a None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:10.912478 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 67/138] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:22:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:22:10.919488 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb32e80d-06ed-452f-8eb9-de0c4341be76 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:10.919846 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb32e80d-06ed-452f-8eb9-de0c4341be76 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:10.920110 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb32e80d-06ed-452f-8eb9-de0c4341be76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:10.948606 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-cb32e80d-06ed-452f-8eb9-de0c4341be76 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:22:10.949829 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-cb32e80d-06ed-452f-8eb9-de0c4341be76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:22:11.049973 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 72/139] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:22:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:22:11.095500 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf161f5-4a15-4df6-af16-7e049c40696a None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:11.232556 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf161f5-4a15-4df6-af16-7e049c40696a None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:11.236171 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a331d2-6573-4728-be98-4034aeb50af7 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:11.236718 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a331d2-6573-4728-be98-4034aeb50af7 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:11.237231 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a331d2-6573-4728-be98-4034aeb50af7 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:11.238830 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 73/140] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:22:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:22:11.244980 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb87dbe-6f46-405b-b80a-ff4c7907a676 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:11.245552 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb87dbe-6f46-405b-b80a-ff4c7907a676 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:11.246044 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb87dbe-6f46-405b-b80a-ff4c7907a676 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:11.257281 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:11.280683 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-8cb87dbe-6f46-405b-b80a-ff4c7907a676 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:22:11.282394 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-8cb87dbe-6f46-405b-b80a-ff4c7907a676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:22:11.292801 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01fb9e8-14c5-4986-bd2d-208906e850f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:11.295060 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01fb9e8-14c5-4986-bd2d-208906e850f1 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:11.295575 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01fb9e8-14c5-4986-bd2d-208906e850f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:11.297609 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01fb9e8-14c5-4986-bd2d-208906e850f1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:11.323624 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01fb9e8-14c5-4986-bd2d-208906e850f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:22:11.328251 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-d01fb9e8-14c5-4986-bd2d-208906e850f1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 15:22:11.328251 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:22:11.328251 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:22:11.328251 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:22:11.328251 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:22:11.328251 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:22:11.328251 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:22:11.328251 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:22:11.328251 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:22:11.328251 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:22:11.328251 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:22:11.328251 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:22:11.328251 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:22:11.328251 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:22:11.328251 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:22:11.328251 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:22:11.329100 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:22:11.329100 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:22:11.329100 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:22:11.329100 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:22:11.329100 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:22:11.329100 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:22:11.329100 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:22:11.329100 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:22:11.329100 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:22:11.329100 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:22:11.329100 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:22:11.329100 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:22:11.329100 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:22:11.329100 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:22:11.329100 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:22:11.329100 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:22:11.329877 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:22:11.329877 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:22:11.329877 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:22:11.329877 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:22:11.329877 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:22:11.329877 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:22:11.329877 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:22:11.329877 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:22:11.329877 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:22:11.329877 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:22:11.329877 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 15:22:11.329877 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:22:11.329877 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 68/141] 10.222.0.99 () {56 vars in 1099 bytes} [Thu Jul 14 15:22:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:22:11.338253 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01fb9e8-14c5-4986-bd2d-208906e850f1 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:11.342089 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:11.417953 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 74/142] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:22:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 15:22:11.459869 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb87dbe-6f46-405b-b80a-ff4c7907a676 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:11.510538 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01fb9e8-14c5-4986-bd2d-208906e850f1 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:11.587915 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bb41c2-7ada-41c8-89b9-aff9c0de9668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:11.588607 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bb41c2-7ada-41c8-89b9-aff9c0de9668 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:11.588901 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bb41c2-7ada-41c8-89b9-aff9c0de9668 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:11.589188 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bb41c2-7ada-41c8-89b9-aff9c0de9668 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:11.590063 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bb41c2-7ada-41c8-89b9-aff9c0de9668 None admin] RBAC: Adding query filter params (name=member) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:22:11.590411 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bb41c2-7ada-41c8-89b9-aff9c0de9668 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:11.592387 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bb41c2-7ada-41c8-89b9-aff9c0de9668 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:11.613959 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 69/143] 10.222.0.99 () {56 vars in 1101 bytes} [Thu Jul 14 15:22:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:11.620406 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bb41c2-7ada-41c8-89b9-aff9c0de9668 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:11.663578 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb87dbe-6f46-405b-b80a-ff4c7907a676 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:11.728447 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:11.739539 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:11.741276 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b797d6f-183d-484a-8ec4-c21738554caf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:11.742122 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b797d6f-183d-484a-8ec4-c21738554caf None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:11.742440 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b797d6f-183d-484a-8ec4-c21738554caf None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:11.744011 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b797d6f-183d-484a-8ec4-c21738554caf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:11.767057 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b797d6f-183d-484a-8ec4-c21738554caf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:22:11.778154 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-7b797d6f-183d-484a-8ec4-c21738554caf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:22:11.778154 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:22:11.778154 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:22:11.778154 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:22:11.778154 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:22:11.778154 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:22:11.778154 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:22:11.778154 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:22:11.778154 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:22:11.778154 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:22:11.778154 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:22:11.778154 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:22:11.778154 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:22:11.778154 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:22:11.778154 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:22:11.778154 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:22:11.779006 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:22:11.779006 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:22:11.779006 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:22:11.779006 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:22:11.779006 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:22:11.779006 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:22:11.779006 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:22:11.779006 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:22:11.779006 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:22:11.779006 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:22:11.779006 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:22:11.779006 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:22:11.779006 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:22:11.779006 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:22:11.779006 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:22:11.779006 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:22:11.779779 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:22:11.779779 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:22:11.779779 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:22:11.779779 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:22:11.779779 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:22:11.779779 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:22:11.779779 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:22:11.779779 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:22:11.779779 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:22:11.779779 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:22:11.779779 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:22:11.779779 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:22:11.781103 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 75/144] 10.222.0.99 () {56 vars in 1096 bytes} [Thu Jul 14 15:22:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 15:22:11.790765 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b797d6f-183d-484a-8ec4-c21738554caf None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:11.821812 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bb41c2-7ada-41c8-89b9-aff9c0de9668 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:11.865125 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9a4e26-81a3-4a99-8f41-dddab61e3964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:11.866322 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9a4e26-81a3-4a99-8f41-dddab61e3964 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:11.866430 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9a4e26-81a3-4a99-8f41-dddab61e3964 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:11.866693 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9a4e26-81a3-4a99-8f41-dddab61e3964 None admin] PATH_INFO: `/v3/users/7f5177fcb1524b918c6662c934105643` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:11.868169 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9a4e26-81a3-4a99-8f41-dddab61e3964 None admin] RBAC: Authorizing `identity:get_user(user_id=7f5177fcb1524b918c6662c934105643)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:11.874911 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9a4e26-81a3-4a99-8f41-dddab61e3964 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:11.874911 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 70/145] 10.222.0.99 () {56 vars in 1177 bytes} [Thu Jul 14 15:22:11 2022] GET /identity/v3/users/7f5177fcb1524b918c6662c934105643 => generated 275 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:11.880615 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9a4e26-81a3-4a99-8f41-dddab61e3964 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:12.009608 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b797d6f-183d-484a-8ec4-c21738554caf None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:12.078006 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fdb5ed-f6ff-4994-82a8-08c5574fc256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:12.078616 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fdb5ed-f6ff-4994-82a8-08c5574fc256 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:12.078908 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fdb5ed-f6ff-4994-82a8-08c5574fc256 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:12.079193 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fdb5ed-f6ff-4994-82a8-08c5574fc256 None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:12.080130 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fdb5ed-f6ff-4994-82a8-08c5574fc256 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:22:12.080447 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fdb5ed-f6ff-4994-82a8-08c5574fc256 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:12.082205 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fdb5ed-f6ff-4994-82a8-08c5574fc256 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:12.099424 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 76/146] 10.222.0.99 () {56 vars in 1099 bytes} [Thu Jul 14 15:22:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:12.106547 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fdb5ed-f6ff-4994-82a8-08c5574fc256 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:12.170558 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9a4e26-81a3-4a99-8f41-dddab61e3964 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:12.202533 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:12.250346 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:12.272399 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3009bc42-1476-4375-9c65-2eebc4c0a4f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:12.273082 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3009bc42-1476-4375-9c65-2eebc4c0a4f3 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:12.273430 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3009bc42-1476-4375-9c65-2eebc4c0a4f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:12.273653 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3009bc42-1476-4375-9c65-2eebc4c0a4f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:12.291027 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fdb5ed-f6ff-4994-82a8-08c5574fc256 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:12.350904 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b5de4b-9ec4-4444-a3b7-ad8c92ba6bff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:12.352541 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b5de4b-9ec4-4444-a3b7-ad8c92ba6bff None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:12.353208 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b5de4b-9ec4-4444-a3b7-ad8c92ba6bff None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:12.353593 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b5de4b-9ec4-4444-a3b7-ad8c92ba6bff None admin] PATH_INFO: `/v3/users/3e33ee1d9fa74c5fa2f58e221ad2d0a8` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:12.355144 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b5de4b-9ec4-4444-a3b7-ad8c92ba6bff None admin] RBAC: Authorizing `identity:get_user(user_id=3e33ee1d9fa74c5fa2f58e221ad2d0a8)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:12.356925 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b5de4b-9ec4-4444-a3b7-ad8c92ba6bff None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:12.358661 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 77/147] 10.222.0.99 () {56 vars in 1177 bytes} [Thu Jul 14 15:22:12 2022] GET /identity/v3/users/3e33ee1d9fa74c5fa2f58e221ad2d0a8 => generated 247 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:22:12.377291 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b5de4b-9ec4-4444-a3b7-ad8c92ba6bff None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:12.443433 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3009bc42-1476-4375-9c65-2eebc4c0a4f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:12.444964 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3009bc42-1476-4375-9c65-2eebc4c0a4f3 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:12.537535 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b5de4b-9ec4-4444-a3b7-ad8c92ba6bff None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:12.562297 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:12.566401 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 71/148] 10.222.0.99 () {58 vars in 1282 bytes} [Thu Jul 14 15:22:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 682 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 15:22:12.568143 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904109ec-ee30-4d5d-8673-4132a6387684 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:12.568584 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904109ec-ee30-4d5d-8673-4132a6387684 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:12.569088 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904109ec-ee30-4d5d-8673-4132a6387684 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:12.571283 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 72/149] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:22:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:22:12.578465 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904109ec-ee30-4d5d-8673-4132a6387684 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:12.590370 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f801e2-9563-4bac-ae2e-d6ba4e1fc5b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:12.591379 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f801e2-9563-4bac-ae2e-d6ba4e1fc5b6 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:12.591935 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f801e2-9563-4bac-ae2e-d6ba4e1fc5b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:12.592435 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f801e2-9563-4bac-ae2e-d6ba4e1fc5b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:12.610295 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:12.679453 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f801e2-9563-4bac-ae2e-d6ba4e1fc5b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:12.681704 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f801e2-9563-4bac-ae2e-d6ba4e1fc5b6 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:12.700199 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904109ec-ee30-4d5d-8673-4132a6387684 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:12.741676 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7118205-9a03-43fd-8c34-ad9e4402b320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:12.743330 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7118205-9a03-43fd-8c34-ad9e4402b320 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:12.743801 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7118205-9a03-43fd-8c34-ad9e4402b320 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:12.744311 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7118205-9a03-43fd-8c34-ad9e4402b320 None admin] PATH_INFO: `/v3/projects/1d9e82a46b1c4b25939e533c26cb44df` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:12.745862 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7118205-9a03-43fd-8c34-ad9e4402b320 None admin] RBAC: Authorizing `identity:get_project(project_id=1d9e82a46b1c4b25939e533c26cb44df)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:12.748200 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7118205-9a03-43fd-8c34-ad9e4402b320 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:12.755544 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 73/150] 10.222.0.99 () {56 vars in 1186 bytes} [Thu Jul 14 15:22:12 2022] GET /identity/v3/projects/1d9e82a46b1c4b25939e533c26cb44df => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:12.757155 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d953d249-1783-405e-bc4f-d24c03a7fb9b None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:12.757531 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d953d249-1783-405e-bc4f-d24c03a7fb9b None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:12.757825 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d953d249-1783-405e-bc4f-d24c03a7fb9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:12.778768 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-d953d249-1783-405e-bc4f-d24c03a7fb9b None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:22:12.780235 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d953d249-1783-405e-bc4f-d24c03a7fb9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:22:12.798588 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 78/151] 10.222.0.99 () {58 vars in 1282 bytes} [Thu Jul 14 15:22:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 15:22:12.804536 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23768823-9ce4-4bad-b5df-dc737cd919cd None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:12.804824 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23768823-9ce4-4bad-b5df-dc737cd919cd None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:12.805150 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23768823-9ce4-4bad-b5df-dc737cd919cd None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:12.807121 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 79/152] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:22:12 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:22:12.808002 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23768823-9ce4-4bad-b5df-dc737cd919cd None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:12.843047 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:12.867004 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 74/153] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:22:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:22:12.868992 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d953d249-1783-405e-bc4f-d24c03a7fb9b None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:12.905288 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:12.939897 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23768823-9ce4-4bad-b5df-dc737cd919cd None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:12.980347 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6b161f-041e-4747-a12e-bc9fd149e4ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:12.981124 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6b161f-041e-4747-a12e-bc9fd149e4ea None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:12.981412 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6b161f-041e-4747-a12e-bc9fd149e4ea None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:12.982024 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6b161f-041e-4747-a12e-bc9fd149e4ea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:12.983417 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6b161f-041e-4747-a12e-bc9fd149e4ea None admin] RBAC: Adding query filter params (role.id=b8c98fddc0fd4436b577fd1d3361bc38, scope.project.id=1d9e82a46b1c4b25939e533c26cb44df, user.id=7f5177fcb1524b918c6662c934105643) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:22:12.983417 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6b161f-041e-4747-a12e-bc9fd149e4ea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:12.984866 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6b161f-041e-4747-a12e-bc9fd149e4ea None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:12.999550 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d953d249-1783-405e-bc4f-d24c03a7fb9b None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:12.999766 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 80/154] 10.222.0.99 () {56 vars in 1374 bytes} [Thu Jul 14 15:22:12 2022] GET /identity/v3/role_assignments?user.id=7f5177fcb1524b918c6662c934105643&scope.project.id=1d9e82a46b1c4b25939e533c26cb44df&role.id=b8c98fddc0fd4436b577fd1d3361bc38 => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:13.001735 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057100ef-d585-427c-b072-531b610958bf None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:13.002042 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057100ef-d585-427c-b072-531b610958bf None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:13.002400 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057100ef-d585-427c-b072-531b610958bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:13.036351 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-057100ef-d585-427c-b072-531b610958bf None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:22:13.038242 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-057100ef-d585-427c-b072-531b610958bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:22:13.055052 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f347a889-5d92-4894-935c-6b2d63e9f0ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:13.058125 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f347a889-5d92-4894-935c-6b2d63e9f0ec None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:13.059620 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f347a889-5d92-4894-935c-6b2d63e9f0ec None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:13.060111 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f347a889-5d92-4894-935c-6b2d63e9f0ec None admin] PATH_INFO: `/v3/projects/1d9e82a46b1c4b25939e533c26cb44df` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:13.063174 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f347a889-5d92-4894-935c-6b2d63e9f0ec None admin] RBAC: Authorizing `identity:get_project(project_id=1d9e82a46b1c4b25939e533c26cb44df)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:13.065077 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f347a889-5d92-4894-935c-6b2d63e9f0ec None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:13.070034 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 75/155] 10.222.0.99 () {56 vars in 1186 bytes} [Thu Jul 14 15:22:12 2022] GET /identity/v3/projects/1d9e82a46b1c4b25939e533c26cb44df => generated 298 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:13.073770 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f347a889-5d92-4894-935c-6b2d63e9f0ec None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:13.201466 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 81/156] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:22:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:22:13.202431 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057100ef-d585-427c-b072-531b610958bf None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:13.251986 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:13.267780 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:13.276865 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f347a889-5d92-4894-935c-6b2d63e9f0ec None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:13.335772 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa3ed2f-4458-4eb4-9594-2678d3ce28f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:13.337321 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa3ed2f-4458-4eb4-9594-2678d3ce28f8 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:13.338179 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa3ed2f-4458-4eb4-9594-2678d3ce28f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:13.338474 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa3ed2f-4458-4eb4-9594-2678d3ce28f8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:13.340024 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa3ed2f-4458-4eb4-9594-2678d3ce28f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:13.341970 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa3ed2f-4458-4eb4-9594-2678d3ce28f8 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:13.344220 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 76/157] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:22:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:13.345014 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa3ed2f-4458-4eb4-9594-2678d3ce28f8 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:13.385513 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057100ef-d585-427c-b072-531b610958bf None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:13.439317 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfece8a-fdd0-469f-8487-a69b416e0363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:13.440308 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfece8a-fdd0-469f-8487-a69b416e0363 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:13.440602 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfece8a-fdd0-469f-8487-a69b416e0363 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:13.440890 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfece8a-fdd0-469f-8487-a69b416e0363 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:13.441757 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfece8a-fdd0-469f-8487-a69b416e0363 None admin] RBAC: Adding query filter params (role.id=d0f6ba1314f14d9f9c816bdb5d0c3aac, scope.project.id=1d9e82a46b1c4b25939e533c26cb44df, user.id=3e33ee1d9fa74c5fa2f58e221ad2d0a8) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:22:13.442330 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfece8a-fdd0-469f-8487-a69b416e0363 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:13.444445 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfece8a-fdd0-469f-8487-a69b416e0363 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:13.453187 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 82/158] 10.222.0.99 () {56 vars in 1374 bytes} [Thu Jul 14 15:22:13 2022] GET /identity/v3/role_assignments?user.id=3e33ee1d9fa74c5fa2f58e221ad2d0a8&scope.project.id=1d9e82a46b1c4b25939e533c26cb44df&role.id=d0f6ba1314f14d9f9c816bdb5d0c3aac => generated 608 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:13.453949 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfece8a-fdd0-469f-8487-a69b416e0363 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:13.500596 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa3ed2f-4458-4eb4-9594-2678d3ce28f8 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:13.547000 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcab248-b4f5-4dbb-a753-2407954787f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:13.548276 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcab248-b4f5-4dbb-a753-2407954787f4 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:13.548828 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcab248-b4f5-4dbb-a753-2407954787f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:13.549381 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcab248-b4f5-4dbb-a753-2407954787f4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:13.565334 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcab248-b4f5-4dbb-a753-2407954787f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:22:13.574420 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-8bcab248-b4f5-4dbb-a753-2407954787f4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 15:22:13.574420 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:22:13.574420 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:22:13.574420 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:22:13.574420 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:22:13.574420 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:22:13.574420 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:22:13.574420 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:22:13.574420 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:22:13.574420 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:22:13.574420 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:22:13.574420 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:22:13.574420 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:22:13.574420 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:22:13.574420 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:22:13.574420 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:22:13.575363 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:22:13.575363 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:22:13.575363 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:22:13.575363 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:22:13.575363 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:22:13.575363 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:22:13.575363 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:22:13.575363 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:22:13.575363 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:22:13.575363 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:22:13.575363 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:22:13.575363 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:22:13.575363 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:22:13.575363 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:22:13.575363 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:22:13.575363 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:22:13.576237 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:22:13.576237 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:22:13.576237 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:22:13.576237 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:22:13.576237 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:22:13.576237 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:22:13.576237 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:22:13.576237 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:22:13.576237 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:22:13.576237 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:22:13.576237 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 15:22:13.576237 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:22:13.578863 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 77/159] 10.222.0.99 () {56 vars in 1114 bytes} [Thu Jul 14 15:22:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:22:13.579845 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9338c31-1c38-4409-bce2-9909c61055f6 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:13.580190 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9338c31-1c38-4409-bce2-9909c61055f6 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:13.580445 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9338c31-1c38-4409-bce2-9909c61055f6 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:13.587333 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 78/160] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:22:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:22:13.588312 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9338c31-1c38-4409-bce2-9909c61055f6 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:13.606788 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:13.631784 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:13.640479 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfece8a-fdd0-469f-8487-a69b416e0363 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:13.683826 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eac13f-7c72-4857-bab8-550bf171b5b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:13.684856 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eac13f-7c72-4857-bab8-550bf171b5b4 None admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:13.685345 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eac13f-7c72-4857-bab8-550bf171b5b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:13.685860 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eac13f-7c72-4857-bab8-550bf171b5b4 None admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:13.686936 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9eac13f-7c72-4857-bab8-550bf171b5b4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:13.689444 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9eac13f-7c72-4857-bab8-550bf171b5b4 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:13.714325 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 83/161] 10.222.0.99 () {60 vars in 1132 bytes} [Thu Jul 14 15:22:13 2022] POST /identity/v3/users => generated 283 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:22:13.715557 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea1a65e-e58c-40b7-b0be-2efd33d3e5b8 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:13.715877 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea1a65e-e58c-40b7-b0be-2efd33d3e5b8 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:13.716166 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea1a65e-e58c-40b7-b0be-2efd33d3e5b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:13.737224 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9338c31-1c38-4409-bce2-9909c61055f6 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:13.753827 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-1ea1a65e-e58c-40b7-b0be-2efd33d3e5b8 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:22:13.755562 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-1ea1a65e-e58c-40b7-b0be-2efd33d3e5b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:22:13.790419 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a21a9e-2e88-4a8b-bc1f-1b509f97594c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:13.791284 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a21a9e-2e88-4a8b-bc1f-1b509f97594c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:13.791698 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a21a9e-2e88-4a8b-bc1f-1b509f97594c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:13.792107 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a21a9e-2e88-4a8b-bc1f-1b509f97594c None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:13.793241 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a21a9e-2e88-4a8b-bc1f-1b509f97594c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:22:13.794117 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a21a9e-2e88-4a8b-bc1f-1b509f97594c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:13.796061 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a21a9e-2e88-4a8b-bc1f-1b509f97594c None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:13.813930 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 79/162] 10.222.0.99 () {56 vars in 1111 bytes} [Thu Jul 14 15:22:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:13.815528 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60289642-377a-4e97-a096-cb368a6bf225 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:13.815958 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60289642-377a-4e97-a096-cb368a6bf225 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:13.816342 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60289642-377a-4e97-a096-cb368a6bf225 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:13.818830 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 80/163] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:22:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:22:13.822738 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60289642-377a-4e97-a096-cb368a6bf225 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:13.855697 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 84/164] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:22:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:22:13.857193 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c989ee03-71dc-4516-833e-e9efa41c42c9 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:13.857493 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c989ee03-71dc-4516-833e-e9efa41c42c9 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:13.857777 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c989ee03-71dc-4516-833e-e9efa41c42c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:13.876741 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-c989ee03-71dc-4516-833e-e9efa41c42c9 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:22:13.878104 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-c989ee03-71dc-4516-833e-e9efa41c42c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:22:13.942331 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:13.954470 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:13.963652 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60289642-377a-4e97-a096-cb368a6bf225 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:13.964839 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 85/165] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:22:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:22:13.965577 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c989ee03-71dc-4516-833e-e9efa41c42c9 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:14.014114 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3408f270-9b64-46d4-aad4-05a0e628ebf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:14.015007 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3408f270-9b64-46d4-aad4-05a0e628ebf8 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:14.015296 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3408f270-9b64-46d4-aad4-05a0e628ebf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:14.015579 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3408f270-9b64-46d4-aad4-05a0e628ebf8 None admin] PATH_INFO: `/v3/users/7f5177fcb1524b918c6662c934105643` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:14.016850 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3408f270-9b64-46d4-aad4-05a0e628ebf8 None admin] RBAC: Authorizing `identity:get_user(user_id=7f5177fcb1524b918c6662c934105643)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:14.018306 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3408f270-9b64-46d4-aad4-05a0e628ebf8 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:14.020294 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 81/166] 10.222.0.99 () {56 vars in 1177 bytes} [Thu Jul 14 15:22:13 2022] GET /identity/v3/users/7f5177fcb1524b918c6662c934105643 => generated 275 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:14.020882 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3408f270-9b64-46d4-aad4-05a0e628ebf8 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:14.107501 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c989ee03-71dc-4516-833e-e9efa41c42c9 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:14.149834 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b8b14d-4f91-4931-aea5-a18a365bd222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:14.150756 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b8b14d-4f91-4931-aea5-a18a365bd222 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:14.151045 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b8b14d-4f91-4931-aea5-a18a365bd222 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:14.151324 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b8b14d-4f91-4931-aea5-a18a365bd222 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:14.159699 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3408f270-9b64-46d4-aad4-05a0e628ebf8 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:14.163797 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b8b14d-4f91-4931-aea5-a18a365bd222 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:22:14.173352 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-52b8b14d-4f91-4931-aea5-a18a365bd222 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 15:22:14.173352 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:22:14.173352 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:22:14.173352 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:22:14.173352 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:22:14.173352 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:22:14.173352 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:22:14.173352 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:22:14.173352 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:22:14.173352 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:22:14.173352 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:22:14.173352 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:22:14.173352 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:22:14.173352 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:22:14.173352 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:22:14.173352 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:22:14.174338 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:22:14.174338 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:22:14.174338 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:22:14.174338 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:22:14.174338 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:22:14.174338 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:22:14.174338 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:22:14.174338 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:22:14.174338 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:22:14.174338 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:22:14.174338 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:22:14.174338 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:22:14.174338 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:22:14.174338 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:22:14.174338 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:22:14.174338 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:22:14.175280 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:22:14.175280 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:22:14.175280 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:22:14.175280 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:22:14.175280 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:22:14.175280 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:22:14.175280 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:22:14.175280 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:22:14.175280 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:22:14.175280 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:22:14.175280 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 15:22:14.175280 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:22:14.177225 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 86/167] 10.222.0.99 () {56 vars in 1099 bytes} [Thu Jul 14 15:22:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:22:14.177899 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b8b14d-4f91-4931-aea5-a18a365bd222 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:14.208516 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398778c4-e360-4527-a5ae-02007407c7f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:14.209328 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398778c4-e360-4527-a5ae-02007407c7f0 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:14.209617 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398778c4-e360-4527-a5ae-02007407c7f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:14.209900 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398778c4-e360-4527-a5ae-02007407c7f0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:14.223410 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398778c4-e360-4527-a5ae-02007407c7f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:22:14.232830 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-398778c4-e360-4527-a5ae-02007407c7f0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 15:22:14.232830 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:22:14.232830 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:22:14.232830 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:22:14.232830 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:22:14.232830 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:22:14.232830 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:22:14.232830 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:22:14.232830 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:22:14.232830 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:22:14.232830 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:22:14.232830 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:22:14.232830 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:22:14.232830 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:22:14.232830 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:22:14.232830 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:22:14.233749 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:22:14.233749 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:22:14.233749 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:22:14.233749 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:22:14.233749 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:22:14.233749 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:22:14.233749 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:22:14.233749 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:22:14.233749 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:22:14.233749 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:22:14.233749 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:22:14.233749 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:22:14.233749 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:22:14.233749 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:22:14.233749 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:22:14.233749 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:22:14.234520 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:22:14.234520 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:22:14.234520 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:22:14.234520 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:22:14.234520 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:22:14.234520 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:22:14.234520 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:22:14.234520 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:22:14.234520 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:22:14.234520 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:22:14.234520 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 15:22:14.234520 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:22:14.234520 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 82/168] 10.222.0.99 () {56 vars in 1099 bytes} [Thu Jul 14 15:22:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:22:14.236002 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398778c4-e360-4527-a5ae-02007407c7f0 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:14.257300 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:14.263106 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:14.312022 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b8b14d-4f91-4931-aea5-a18a365bd222 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:14.355503 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e99828-cd45-4a86-b1fa-97dc45975833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:14.357357 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e99828-cd45-4a86-b1fa-97dc45975833 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:14.357796 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e99828-cd45-4a86-b1fa-97dc45975833 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:14.358237 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e99828-cd45-4a86-b1fa-97dc45975833 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:14.393701 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398778c4-e360-4527-a5ae-02007407c7f0 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:14.430613 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556f88f5-5c0a-4784-a321-dc2b873e873f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:14.431535 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556f88f5-5c0a-4784-a321-dc2b873e873f None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:14.431998 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556f88f5-5c0a-4784-a321-dc2b873e873f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:14.432423 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556f88f5-5c0a-4784-a321-dc2b873e873f None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:14.433326 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556f88f5-5c0a-4784-a321-dc2b873e873f None admin] RBAC: Adding query filter params (name=member) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:22:14.433881 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556f88f5-5c0a-4784-a321-dc2b873e873f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:14.435416 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556f88f5-5c0a-4784-a321-dc2b873e873f None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:14.444992 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 83/169] 10.222.0.99 () {56 vars in 1101 bytes} [Thu Jul 14 15:22:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:14.446594 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556f88f5-5c0a-4784-a321-dc2b873e873f None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:14.491652 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e99828-cd45-4a86-b1fa-97dc45975833 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:14.495591 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e99828-cd45-4a86-b1fa-97dc45975833 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:14.561426 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:14.606510 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556f88f5-5c0a-4784-a321-dc2b873e873f None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:14.608364 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:14.674848 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db7a371-4253-44b1-95f3-7bbfa995f54a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:14.675530 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db7a371-4253-44b1-95f3-7bbfa995f54a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:14.675850 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db7a371-4253-44b1-95f3-7bbfa995f54a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:14.676133 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db7a371-4253-44b1-95f3-7bbfa995f54a None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:14.676903 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db7a371-4253-44b1-95f3-7bbfa995f54a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:22:14.677223 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db7a371-4253-44b1-95f3-7bbfa995f54a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:14.678636 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db7a371-4253-44b1-95f3-7bbfa995f54a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:14.686416 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 87/170] 10.222.0.99 () {58 vars in 1282 bytes} [Thu Jul 14 15:22:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 509 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 15:22:14.687293 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e99828-cd45-4a86-b1fa-97dc45975833 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:14.695577 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 84/171] 10.222.0.99 () {56 vars in 1101 bytes} [Thu Jul 14 15:22:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:14.696994 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db7a371-4253-44b1-95f3-7bbfa995f54a None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:14.829215 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e99828-cd45-4a86-b1fa-97dc45975833 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:14.872268 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171da6cd-d4b2-47de-83e8-3bd7d2ced59d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:14.874069 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171da6cd-d4b2-47de-83e8-3bd7d2ced59d None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:14.874613 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171da6cd-d4b2-47de-83e8-3bd7d2ced59d None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:14.875090 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171da6cd-d4b2-47de-83e8-3bd7d2ced59d None admin] PATH_INFO: `/v3/users/7f5177fcb1524b918c6662c934105643` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:14.876818 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171da6cd-d4b2-47de-83e8-3bd7d2ced59d None admin] RBAC: Authorizing `identity:get_user(user_id=7f5177fcb1524b918c6662c934105643)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:14.878561 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171da6cd-d4b2-47de-83e8-3bd7d2ced59d None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:14.880754 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 88/172] 10.222.0.99 () {56 vars in 1177 bytes} [Thu Jul 14 15:22:14 2022] GET /identity/v3/users/7f5177fcb1524b918c6662c934105643 => generated 275 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:14.882316 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171da6cd-d4b2-47de-83e8-3bd7d2ced59d None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:14.913714 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:14.939245 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db7a371-4253-44b1-95f3-7bbfa995f54a None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:15.009738 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada748bc-f2fb-4d29-b75e-25afbf663faf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:15.010678 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada748bc-f2fb-4d29-b75e-25afbf663faf None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:15.011047 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada748bc-f2fb-4d29-b75e-25afbf663faf None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:15.015564 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada748bc-f2fb-4d29-b75e-25afbf663faf None admin] PATH_INFO: `/v3/projects/1d9e82a46b1c4b25939e533c26cb44df` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:15.016939 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada748bc-f2fb-4d29-b75e-25afbf663faf None admin] RBAC: Authorizing `identity:get_project(project_id=1d9e82a46b1c4b25939e533c26cb44df)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:15.018407 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada748bc-f2fb-4d29-b75e-25afbf663faf None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:15.020159 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 85/173] 10.222.0.99 () {56 vars in 1186 bytes} [Thu Jul 14 15:22:14 2022] GET /identity/v3/projects/1d9e82a46b1c4b25939e533c26cb44df => generated 298 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:15.023082 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada748bc-f2fb-4d29-b75e-25afbf663faf None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:15.046604 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:15.065256 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171da6cd-d4b2-47de-83e8-3bd7d2ced59d None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:15.105771 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448b8b1b-e8e9-4950-a50d-517c6dfaa2bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:15.106643 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448b8b1b-e8e9-4950-a50d-517c6dfaa2bd None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:15.106928 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448b8b1b-e8e9-4950-a50d-517c6dfaa2bd None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:15.107205 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448b8b1b-e8e9-4950-a50d-517c6dfaa2bd None admin] PATH_INFO: `/v3/users/c83c274d5da64fed90c95ef77dcaa254` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:15.108801 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448b8b1b-e8e9-4950-a50d-517c6dfaa2bd None admin] RBAC: Authorizing `identity:get_user(user_id=c83c274d5da64fed90c95ef77dcaa254)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:15.110327 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448b8b1b-e8e9-4950-a50d-517c6dfaa2bd None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:15.112334 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 89/174] 10.222.0.99 () {56 vars in 1177 bytes} [Thu Jul 14 15:22:14 2022] GET /identity/v3/users/c83c274d5da64fed90c95ef77dcaa254 => generated 289 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:15.113105 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448b8b1b-e8e9-4950-a50d-517c6dfaa2bd None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:15.235111 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada748bc-f2fb-4d29-b75e-25afbf663faf None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:15.279988 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:15.296860 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448b8b1b-e8e9-4950-a50d-517c6dfaa2bd None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:15.309264 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc453ae-54fc-40a7-9f00-a8adc801b04c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:15.310499 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc453ae-54fc-40a7-9f00-a8adc801b04c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:15.310785 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc453ae-54fc-40a7-9f00-a8adc801b04c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:15.311064 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc453ae-54fc-40a7-9f00-a8adc801b04c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:15.363503 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4f7ea6-6609-4a88-b040-f6b27ac31279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:15.364113 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4f7ea6-6609-4a88-b040-f6b27ac31279 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:15.364396 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4f7ea6-6609-4a88-b040-f6b27ac31279 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:15.364672 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4f7ea6-6609-4a88-b040-f6b27ac31279 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:15.365515 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4f7ea6-6609-4a88-b040-f6b27ac31279 None admin] RBAC: Adding query filter params (role.id=3627b4f45c8c4f41a7def6b15f1b8225, scope.project.id=1d9e82a46b1c4b25939e533c26cb44df, user.id=7f5177fcb1524b918c6662c934105643) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:22:15.365825 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4f7ea6-6609-4a88-b040-f6b27ac31279 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:15.367279 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4f7ea6-6609-4a88-b040-f6b27ac31279 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:15.386245 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 90/175] 10.222.0.99 () {56 vars in 1374 bytes} [Thu Jul 14 15:22:15 2022] GET /identity/v3/role_assignments?user.id=7f5177fcb1524b918c6662c934105643&scope.project.id=1d9e82a46b1c4b25939e533c26cb44df&role.id=3627b4f45c8c4f41a7def6b15f1b8225 => generated 608 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:15.387776 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4f7ea6-6609-4a88-b040-f6b27ac31279 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:15.427273 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc453ae-54fc-40a7-9f00-a8adc801b04c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:15.428804 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc453ae-54fc-40a7-9f00-a8adc801b04c None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:15.443112 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:15.551505 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 86/176] 10.222.0.99 () {58 vars in 1282 bytes} [Thu Jul 14 15:22:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 531 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 15:22:15.558506 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc453ae-54fc-40a7-9f00-a8adc801b04c None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:15.672057 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc453ae-54fc-40a7-9f00-a8adc801b04c None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:15.684031 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:15.705964 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4f7ea6-6609-4a88-b040-f6b27ac31279 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:15.719780 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a959dd8d-9dd5-49c7-ae2b-98f774feebac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:15.720655 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a959dd8d-9dd5-49c7-ae2b-98f774feebac None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:15.720935 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a959dd8d-9dd5-49c7-ae2b-98f774feebac None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:15.721214 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a959dd8d-9dd5-49c7-ae2b-98f774feebac None admin] PATH_INFO: `/v3/projects/f966b0d68ba448648d66bce40b1605eb` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:15.724483 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a959dd8d-9dd5-49c7-ae2b-98f774feebac None admin] RBAC: Authorizing `identity:get_project(project_id=f966b0d68ba448648d66bce40b1605eb)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:15.725923 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a959dd8d-9dd5-49c7-ae2b-98f774feebac None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:15.728393 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 87/177] 10.222.0.99 () {56 vars in 1186 bytes} [Thu Jul 14 15:22:15 2022] GET /identity/v3/projects/f966b0d68ba448648d66bce40b1605eb => generated 312 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:22:15.734729 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a959dd8d-9dd5-49c7-ae2b-98f774feebac None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:15.779095 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af791f7-8f4a-4a74-8b87-eca2bd6d052a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:15.780069 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af791f7-8f4a-4a74-8b87-eca2bd6d052a None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:15.780349 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af791f7-8f4a-4a74-8b87-eca2bd6d052a None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:15.780628 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af791f7-8f4a-4a74-8b87-eca2bd6d052a None admin] PATH_INFO: `/v3/projects/1d9e82a46b1c4b25939e533c26cb44df` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:15.781859 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af791f7-8f4a-4a74-8b87-eca2bd6d052a None admin] RBAC: Authorizing `identity:get_project(project_id=1d9e82a46b1c4b25939e533c26cb44df)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:15.783399 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af791f7-8f4a-4a74-8b87-eca2bd6d052a None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:15.792793 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 91/178] 10.222.0.99 () {56 vars in 1186 bytes} [Thu Jul 14 15:22:15 2022] GET /identity/v3/projects/1d9e82a46b1c4b25939e533c26cb44df => generated 298 bytes in 406 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:15.807163 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af791f7-8f4a-4a74-8b87-eca2bd6d052a None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:15.856011 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:15.878373 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a959dd8d-9dd5-49c7-ae2b-98f774feebac None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:15.922249 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844400c7-672c-4fbd-9b9f-8eb343150619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:15.923298 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844400c7-672c-4fbd-9b9f-8eb343150619 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:15.923585 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844400c7-672c-4fbd-9b9f-8eb343150619 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:15.924040 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844400c7-672c-4fbd-9b9f-8eb343150619 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:15.925548 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844400c7-672c-4fbd-9b9f-8eb343150619 None admin] RBAC: Adding query filter params (role.id=b8c98fddc0fd4436b577fd1d3361bc38, scope.project.id=f966b0d68ba448648d66bce40b1605eb, user.id=7f5177fcb1524b918c6662c934105643) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:22:15.925869 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844400c7-672c-4fbd-9b9f-8eb343150619 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:15.927329 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844400c7-672c-4fbd-9b9f-8eb343150619 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:15.934999 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 88/179] 10.222.0.99 () {56 vars in 1374 bytes} [Thu Jul 14 15:22:15 2022] GET /identity/v3/role_assignments?user.id=7f5177fcb1524b918c6662c934105643&scope.project.id=f966b0d68ba448648d66bce40b1605eb&role.id=b8c98fddc0fd4436b577fd1d3361bc38 => generated 608 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:16.025984 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af791f7-8f4a-4a74-8b87-eca2bd6d052a None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:16.071030 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a94c0a-6955-4858-a89f-bc31d417467d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:16.072528 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a94c0a-6955-4858-a89f-bc31d417467d None admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:16.072821 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a94c0a-6955-4858-a89f-bc31d417467d None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:16.073103 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a94c0a-6955-4858-a89f-bc31d417467d None admin] PATH_INFO: `/v3/projects/1d9e82a46b1c4b25939e533c26cb44df/users/c83c274d5da64fed90c95ef77dcaa254/roles/cc5c3091b148423684e5e966150bc6c6` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:16.075233 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a94c0a-6955-4858-a89f-bc31d417467d None admin] RBAC: Authorizing `identity:create_grant(project_id=1d9e82a46b1c4b25939e533c26cb44df, user_id=c83c274d5da64fed90c95ef77dcaa254, role_id=cc5c3091b148423684e5e966150bc6c6)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:16.076783 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a94c0a-6955-4858-a89f-bc31d417467d None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:16.095503 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 92/180] 10.222.0.99 () {58 vars in 1439 bytes} [Thu Jul 14 15:22:15 2022] PUT /identity/v3/projects/1d9e82a46b1c4b25939e533c26cb44df/users/c83c274d5da64fed90c95ef77dcaa254/roles/cc5c3091b148423684e5e966150bc6c6 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 15:22:16.283632 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a000df5e-7118-4762-9f92-db56798c08ca None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:16.284112 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a000df5e-7118-4762-9f92-db56798c08ca None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:16.284112 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a000df5e-7118-4762-9f92-db56798c08ca None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:16.285819 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 89/181] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:22:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:22:16.291652 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be64b5f-ccb1-48c6-a715-029dc2e25b62 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:16.292568 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be64b5f-ccb1-48c6-a715-029dc2e25b62 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:16.293056 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be64b5f-ccb1-48c6-a715-029dc2e25b62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:16.312230 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-4be64b5f-ccb1-48c6-a715-029dc2e25b62 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:22:16.313775 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-4be64b5f-ccb1-48c6-a715-029dc2e25b62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:22:16.391627 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 93/182] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:22:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:22:16.416666 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a000df5e-7118-4762-9f92-db56798c08ca None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:16.467006 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:16.529196 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a000df5e-7118-4762-9f92-db56798c08ca None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:16.567679 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3a4fb2-132b-401f-a461-7013a052227b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:16.568405 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3a4fb2-132b-401f-a461-7013a052227b None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:16.568685 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3a4fb2-132b-401f-a461-7013a052227b None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:16.568959 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3a4fb2-132b-401f-a461-7013a052227b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:16.580526 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3a4fb2-132b-401f-a461-7013a052227b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:22:16.588539 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-3c3a4fb2-132b-401f-a461-7013a052227b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:22:16.588539 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:22:16.588539 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:22:16.588539 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:22:16.588539 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:22:16.588539 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:22:16.588539 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:22:16.588539 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:22:16.588539 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:22:16.588539 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:22:16.588539 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:22:16.588539 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:22:16.588539 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:22:16.588539 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:22:16.588539 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:22:16.588539 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:22:16.589412 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:22:16.589412 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:22:16.589412 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:22:16.589412 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:22:16.589412 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:22:16.589412 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:22:16.589412 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:22:16.589412 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:22:16.589412 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:22:16.589412 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:22:16.589412 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:22:16.589412 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:22:16.589412 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:22:16.589412 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:22:16.589412 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:22:16.589412 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:22:16.590182 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:22:16.590182 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:22:16.590182 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:22:16.590182 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:22:16.590182 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:22:16.590182 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:22:16.590182 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:22:16.590182 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:22:16.590182 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:22:16.590182 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:22:16.590182 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:22:16.590182 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:22:16.590182 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 90/183] 10.222.0.99 () {56 vars in 1096 bytes} [Thu Jul 14 15:22:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:22:16.595843 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be64b5f-ccb1-48c6-a715-029dc2e25b62 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:16.660026 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:16.703329 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be64b5f-ccb1-48c6-a715-029dc2e25b62 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:16.706616 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6326e4-9967-4806-b2df-539d68c3383a None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:16.706917 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6326e4-9967-4806-b2df-539d68c3383a None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:16.707204 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6326e4-9967-4806-b2df-539d68c3383a None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:16.708641 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 91/184] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:22:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:22:16.714626 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938a0c83-a0a8-4335-98ed-a84159b9cd0b None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:16.714922 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938a0c83-a0a8-4335-98ed-a84159b9cd0b None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:16.715208 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938a0c83-a0a8-4335-98ed-a84159b9cd0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:16.734624 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-938a0c83-a0a8-4335-98ed-a84159b9cd0b None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:22:16.735886 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-938a0c83-a0a8-4335-98ed-a84159b9cd0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:22:16.743169 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62d6466-9cc7-41c4-8483-ba0fc2c749c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:16.743902 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62d6466-9cc7-41c4-8483-ba0fc2c749c7 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:16.744193 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62d6466-9cc7-41c4-8483-ba0fc2c749c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:16.745040 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62d6466-9cc7-41c4-8483-ba0fc2c749c7 None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:16.745810 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62d6466-9cc7-41c4-8483-ba0fc2c749c7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:22:16.746117 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62d6466-9cc7-41c4-8483-ba0fc2c749c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:16.748026 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62d6466-9cc7-41c4-8483-ba0fc2c749c7 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:16.760093 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 94/185] 10.222.0.99 () {56 vars in 1099 bytes} [Thu Jul 14 15:22:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:16.765824 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62d6466-9cc7-41c4-8483-ba0fc2c749c7 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:16.819845 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 92/186] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:22:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:22:16.821263 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22bd3f3-2f0e-4744-9548-a64d5b56b73f None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:16.821629 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22bd3f3-2f0e-4744-9548-a64d5b56b73f None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:16.821911 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22bd3f3-2f0e-4744-9548-a64d5b56b73f None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:16.823440 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 93/187] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:22:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:22:16.828504 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e21355-b87a-4d1a-85d0-809b4dd32410 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:16.828747 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e21355-b87a-4d1a-85d0-809b4dd32410 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:16.829033 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e21355-b87a-4d1a-85d0-809b4dd32410 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:16.846428 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-57e21355-b87a-4d1a-85d0-809b4dd32410 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:22:16.847694 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-57e21355-b87a-4d1a-85d0-809b4dd32410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:22:16.855507 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:16.873952 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:16.882975 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62d6466-9cc7-41c4-8483-ba0fc2c749c7 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:16.921640 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce02e484-66c4-4e6d-be1c-c5e117b4a475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:16.922818 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce02e484-66c4-4e6d-be1c-c5e117b4a475 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:16.923265 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce02e484-66c4-4e6d-be1c-c5e117b4a475 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:16.923731 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce02e484-66c4-4e6d-be1c-c5e117b4a475 None admin] PATH_INFO: `/v3/users/6ddb9f84c97f418a88ac32e44ba83caa` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:16.926381 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 94/188] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:22:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 15:22:16.927262 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e21355-b87a-4d1a-85d0-809b4dd32410 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:16.950861 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce02e484-66c4-4e6d-be1c-c5e117b4a475 None admin] RBAC: Authorizing `identity:get_user(user_id=6ddb9f84c97f418a88ac32e44ba83caa)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:16.952698 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce02e484-66c4-4e6d-be1c-c5e117b4a475 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:16.954860 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 95/189] 10.222.0.99 () {56 vars in 1177 bytes} [Thu Jul 14 15:22:16 2022] GET /identity/v3/users/6ddb9f84c97f418a88ac32e44ba83caa => generated 283 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:22:16.960407 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce02e484-66c4-4e6d-be1c-c5e117b4a475 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:17.048802 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e21355-b87a-4d1a-85d0-809b4dd32410 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:17.073649 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce02e484-66c4-4e6d-be1c-c5e117b4a475 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:17.082845 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4c47c2-ad75-47cd-b311-3aa18a93a5f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:17.083690 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4c47c2-ad75-47cd-b311-3aa18a93a5f3 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:17.083978 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4c47c2-ad75-47cd-b311-3aa18a93a5f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:17.084259 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4c47c2-ad75-47cd-b311-3aa18a93a5f3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:17.096271 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4c47c2-ad75-47cd-b311-3aa18a93a5f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:22:17.105284 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-ee4c47c2-ad75-47cd-b311-3aa18a93a5f3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 15:22:17.105284 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:22:17.105284 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:22:17.105284 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:22:17.105284 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:22:17.105284 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:22:17.105284 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:22:17.105284 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:22:17.105284 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:22:17.105284 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:22:17.105284 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:22:17.105284 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:22:17.105284 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:22:17.105284 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:22:17.105284 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:22:17.105284 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:22:17.106232 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:22:17.106232 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:22:17.106232 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:22:17.106232 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:22:17.106232 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:22:17.106232 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:22:17.106232 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:22:17.106232 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:22:17.106232 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:22:17.106232 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:22:17.106232 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:22:17.106232 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:22:17.106232 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:22:17.106232 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:22:17.106232 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:22:17.106232 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:22:17.112323 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:22:17.112323 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:22:17.112323 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:22:17.112323 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:22:17.112323 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:22:17.112323 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:22:17.112323 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:22:17.112323 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:22:17.112323 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:22:17.112323 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:22:17.112323 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 15:22:17.112323 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:22:17.112323 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 95/190] 10.222.0.99 () {56 vars in 1114 bytes} [Thu Jul 14 15:22:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:22:17.112323 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4c47c2-ad75-47cd-b311-3aa18a93a5f3 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:17.117578 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b33d194-9fce-434b-918a-02778e6a6f7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:17.119118 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b33d194-9fce-434b-918a-02778e6a6f7d None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:17.119581 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b33d194-9fce-434b-918a-02778e6a6f7d None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:17.120182 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b33d194-9fce-434b-918a-02778e6a6f7d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:17.121347 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b33d194-9fce-434b-918a-02778e6a6f7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:17.122929 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b33d194-9fce-434b-918a-02778e6a6f7d None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:17.124843 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 96/191] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:22:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:22:17.125621 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b33d194-9fce-434b-918a-02778e6a6f7d None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:17.128893 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:17.142170 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:17.231147 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4c47c2-ad75-47cd-b311-3aa18a93a5f3 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:17.237068 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b33d194-9fce-434b-918a-02778e6a6f7d None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:17.268511 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127436e0-250b-4828-ba9a-d10ad7405f40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:17.269170 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127436e0-250b-4828-ba9a-d10ad7405f40 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:17.269465 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127436e0-250b-4828-ba9a-d10ad7405f40 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:17.269747 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127436e0-250b-4828-ba9a-d10ad7405f40 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:17.275752 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa8f591-77d4-418d-a67c-00f7d3a05b31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:17.276434 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa8f591-77d4-418d-a67c-00f7d3a05b31 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:17.276721 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa8f591-77d4-418d-a67c-00f7d3a05b31 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:17.277000 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa8f591-77d4-418d-a67c-00f7d3a05b31 None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:17.277756 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa8f591-77d4-418d-a67c-00f7d3a05b31 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:22:17.278066 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa8f591-77d4-418d-a67c-00f7d3a05b31 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:17.279920 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa8f591-77d4-418d-a67c-00f7d3a05b31 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:17.294121 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 97/192] 10.222.0.99 () {56 vars in 1111 bytes} [Thu Jul 14 15:22:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:17.294121 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa8f591-77d4-418d-a67c-00f7d3a05b31 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:17.350334 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127436e0-250b-4828-ba9a-d10ad7405f40 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:17.351754 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127436e0-250b-4828-ba9a-d10ad7405f40 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:17.366020 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:17.376313 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:17.413839 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa8f591-77d4-418d-a67c-00f7d3a05b31 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:17.448318 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e191fc9-a043-4387-841f-150a20c865f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:17.449007 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e191fc9-a043-4387-841f-150a20c865f9 None admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:17.449292 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e191fc9-a043-4387-841f-150a20c865f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:17.449571 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e191fc9-a043-4387-841f-150a20c865f9 None admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:17.450284 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e191fc9-a043-4387-841f-150a20c865f9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:17.451750 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e191fc9-a043-4387-841f-150a20c865f9 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:17.469907 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 96/193] 10.222.0.99 () {58 vars in 1282 bytes} [Thu Jul 14 15:22:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 15:22:17.469907 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127436e0-250b-4828-ba9a-d10ad7405f40 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:17.478500 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 98/194] 10.222.0.99 () {60 vars in 1132 bytes} [Thu Jul 14 15:22:17 2022] POST /identity/v3/users => generated 297 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:22:17.479624 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e191fc9-a043-4387-841f-150a20c865f9 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:17.612368 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127436e0-250b-4828-ba9a-d10ad7405f40 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:17.639590 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e191fc9-a043-4387-841f-150a20c865f9 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:17.640911 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:17.661496 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabbb7ee-333e-4601-8275-832c8da8ddc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:17.662731 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabbb7ee-333e-4601-8275-832c8da8ddc2 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:17.663246 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabbb7ee-333e-4601-8275-832c8da8ddc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:17.663769 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:17.664013 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabbb7ee-333e-4601-8275-832c8da8ddc2 None admin] PATH_INFO: `/v3/users/6ddb9f84c97f418a88ac32e44ba83caa` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:17.665649 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabbb7ee-333e-4601-8275-832c8da8ddc2 None admin] RBAC: Authorizing `identity:get_user(user_id=6ddb9f84c97f418a88ac32e44ba83caa)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:17.667402 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabbb7ee-333e-4601-8275-832c8da8ddc2 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:17.669727 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 97/195] 10.222.0.99 () {56 vars in 1177 bytes} [Thu Jul 14 15:22:17 2022] GET /identity/v3/users/6ddb9f84c97f418a88ac32e44ba83caa => generated 283 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:17.675241 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabbb7ee-333e-4601-8275-832c8da8ddc2 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:17.684320 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f27336-8316-411b-8c8a-7bcde0db5d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:17.685274 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f27336-8316-411b-8c8a-7bcde0db5d31 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:17.685737 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f27336-8316-411b-8c8a-7bcde0db5d31 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:17.686022 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f27336-8316-411b-8c8a-7bcde0db5d31 None admin] PATH_INFO: `/v3/projects/925bf9baf40d4a888a9d633d1deb5134` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:17.687258 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f27336-8316-411b-8c8a-7bcde0db5d31 None admin] RBAC: Authorizing `identity:get_project(project_id=925bf9baf40d4a888a9d633d1deb5134)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:17.688761 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f27336-8316-411b-8c8a-7bcde0db5d31 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:17.690521 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 99/196] 10.222.0.99 () {56 vars in 1186 bytes} [Thu Jul 14 15:22:17 2022] GET /identity/v3/projects/925bf9baf40d4a888a9d633d1deb5134 => generated 302 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:17.697603 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f27336-8316-411b-8c8a-7bcde0db5d31 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:17.807115 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabbb7ee-333e-4601-8275-832c8da8ddc2 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:17.845385 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f27336-8316-411b-8c8a-7bcde0db5d31 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:17.854962 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9a8647-cd33-48cf-a01c-0ed24c11edde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:17.855760 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9a8647-cd33-48cf-a01c-0ed24c11edde None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:17.856056 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9a8647-cd33-48cf-a01c-0ed24c11edde None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:17.856367 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9a8647-cd33-48cf-a01c-0ed24c11edde None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:17.882984 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cea03b5-722c-49c2-baca-101f249ba61d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:17.883580 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cea03b5-722c-49c2-baca-101f249ba61d None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:17.883942 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cea03b5-722c-49c2-baca-101f249ba61d None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:17.884325 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cea03b5-722c-49c2-baca-101f249ba61d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:17.885225 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cea03b5-722c-49c2-baca-101f249ba61d None admin] RBAC: Adding query filter params (role.id=d0f6ba1314f14d9f9c816bdb5d0c3aac, scope.project.id=925bf9baf40d4a888a9d633d1deb5134, user.id=6ddb9f84c97f418a88ac32e44ba83caa) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:22:17.885727 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cea03b5-722c-49c2-baca-101f249ba61d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:17.887520 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cea03b5-722c-49c2-baca-101f249ba61d None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:17.899661 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 100/197] 10.222.0.99 () {56 vars in 1374 bytes} [Thu Jul 14 15:22:17 2022] GET /identity/v3/role_assignments?user.id=6ddb9f84c97f418a88ac32e44ba83caa&scope.project.id=925bf9baf40d4a888a9d633d1deb5134&role.id=d0f6ba1314f14d9f9c816bdb5d0c3aac => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:17.929543 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:17.945625 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9a8647-cd33-48cf-a01c-0ed24c11edde None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:17.948279 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9a8647-cd33-48cf-a01c-0ed24c11edde None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:18.081112 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 98/198] 10.222.0.99 () {58 vars in 1282 bytes} [Thu Jul 14 15:22:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 14 15:22:18.086836 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cea03b5-722c-49c2-baca-101f249ba61d None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:18.155298 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:18.219403 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cea03b5-722c-49c2-baca-101f249ba61d None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:18.261852 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824faa87-5521-4a43-86c8-03a3c69051a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:18.262737 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824faa87-5521-4a43-86c8-03a3c69051a2 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:18.263028 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824faa87-5521-4a43-86c8-03a3c69051a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:18.263311 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824faa87-5521-4a43-86c8-03a3c69051a2 None admin] PATH_INFO: `/v3/projects/925bf9baf40d4a888a9d633d1deb5134` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:18.264573 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824faa87-5521-4a43-86c8-03a3c69051a2 None admin] RBAC: Authorizing `identity:get_project(project_id=925bf9baf40d4a888a9d633d1deb5134)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:18.266045 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824faa87-5521-4a43-86c8-03a3c69051a2 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:18.268102 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 101/199] 10.222.0.99 () {56 vars in 1186 bytes} [Thu Jul 14 15:22:18 2022] GET /identity/v3/projects/925bf9baf40d4a888a9d633d1deb5134 => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:18.274265 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9a8647-cd33-48cf-a01c-0ed24c11edde None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:18.405650 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:18.420298 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9a8647-cd33-48cf-a01c-0ed24c11edde None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:18.456264 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fbabe2-e9b4-408c-9c60-a27198134a29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:18.457285 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fbabe2-e9b4-408c-9c60-a27198134a29 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:18.457771 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fbabe2-e9b4-408c-9c60-a27198134a29 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:18.458249 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fbabe2-e9b4-408c-9c60-a27198134a29 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:18.459296 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fbabe2-e9b4-408c-9c60-a27198134a29 None admin] RBAC: Adding query filter params (role.id=3627b4f45c8c4f41a7def6b15f1b8225, scope.project.id=925bf9baf40d4a888a9d633d1deb5134, user.id=6ddb9f84c97f418a88ac32e44ba83caa) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:22:18.459854 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fbabe2-e9b4-408c-9c60-a27198134a29 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:18.461493 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fbabe2-e9b4-408c-9c60-a27198134a29 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:18.468175 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 99/200] 10.222.0.99 () {56 vars in 1374 bytes} [Thu Jul 14 15:22:18 2022] GET /identity/v3/role_assignments?user.id=6ddb9f84c97f418a88ac32e44ba83caa&scope.project.id=925bf9baf40d4a888a9d633d1deb5134&role.id=3627b4f45c8c4f41a7def6b15f1b8225 => generated 259 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:18.513840 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cec49be-8756-4713-9b3b-8251e9f84323 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:18.514937 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cec49be-8756-4713-9b3b-8251e9f84323 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:18.515456 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cec49be-8756-4713-9b3b-8251e9f84323 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:18.517360 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 102/201] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:22:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:22:18.523086 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaf7749-a7d4-4604-b748-5987ea5ee9f4 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:18.523702 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaf7749-a7d4-4604-b748-5987ea5ee9f4 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:18.524183 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaf7749-a7d4-4604-b748-5987ea5ee9f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:18.543110 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-ceaf7749-a7d4-4604-b748-5987ea5ee9f4 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:22:18.544691 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ceaf7749-a7d4-4604-b748-5987ea5ee9f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:22:18.621759 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 100/202] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:22:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:22:18.646559 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cec49be-8756-4713-9b3b-8251e9f84323 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:18.783343 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cec49be-8756-4713-9b3b-8251e9f84323 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:18.811144 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:18.829020 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa35bc7-2284-4444-848a-b2dd79086f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:18.829834 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa35bc7-2284-4444-848a-b2dd79086f48 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:18.830125 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa35bc7-2284-4444-848a-b2dd79086f48 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:18.830405 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa35bc7-2284-4444-848a-b2dd79086f48 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:18.843316 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa35bc7-2284-4444-848a-b2dd79086f48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:22:18.851171 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-caa35bc7-2284-4444-848a-b2dd79086f48 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 15:22:18.851171 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:22:18.851171 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:22:18.851171 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:22:18.851171 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:22:18.851171 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:22:18.851171 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:22:18.851171 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:22:18.851171 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:22:18.851171 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:22:18.851171 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:22:18.851171 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:22:18.851171 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:22:18.851171 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:22:18.851171 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:22:18.851171 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:22:18.852044 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:22:18.852044 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:22:18.852044 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:22:18.852044 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:22:18.852044 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:22:18.852044 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:22:18.852044 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:22:18.852044 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:22:18.852044 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:22:18.852044 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:22:18.852044 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:22:18.852044 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:22:18.852044 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:22:18.852044 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:22:18.852044 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:22:18.852044 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:22:18.853376 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:22:18.853376 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:22:18.853376 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:22:18.853376 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:22:18.853376 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:22:18.853376 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:22:18.853376 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:22:18.853376 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:22:18.853376 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:22:18.853376 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:22:18.853376 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 15:22:18.853376 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:22:18.853376 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 103/203] 10.222.0.99 () {56 vars in 1099 bytes} [Thu Jul 14 15:22:18 2022] GET /identity/v3/roles/reader => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:22:18.860872 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaf7749-a7d4-4604-b748-5987ea5ee9f4 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:19.000850 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:19.022112 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaf7749-a7d4-4604-b748-5987ea5ee9f4 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:19.064632 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23810169-9ee8-42fa-ad8a-4d1e090e9999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:19.065230 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23810169-9ee8-42fa-ad8a-4d1e090e9999 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:19.065512 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23810169-9ee8-42fa-ad8a-4d1e090e9999 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:19.065789 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23810169-9ee8-42fa-ad8a-4d1e090e9999 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:19.066546 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23810169-9ee8-42fa-ad8a-4d1e090e9999 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:22:19.066848 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23810169-9ee8-42fa-ad8a-4d1e090e9999 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:19.068272 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23810169-9ee8-42fa-ad8a-4d1e090e9999 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:19.084216 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 101/204] 10.222.0.99 () {56 vars in 1101 bytes} [Thu Jul 14 15:22:18 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:19.090478 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa35bc7-2284-4444-848a-b2dd79086f48 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:19.258863 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa35bc7-2284-4444-848a-b2dd79086f48 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:19.307434 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ac5f3e-1c98-470a-89d7-253284364e98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:19.308699 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ac5f3e-1c98-470a-89d7-253284364e98 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:19.309198 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ac5f3e-1c98-470a-89d7-253284364e98 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:19.309696 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ac5f3e-1c98-470a-89d7-253284364e98 None admin] PATH_INFO: `/v3/users/c83c274d5da64fed90c95ef77dcaa254` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:19.311154 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ac5f3e-1c98-470a-89d7-253284364e98 None admin] RBAC: Authorizing `identity:get_user(user_id=c83c274d5da64fed90c95ef77dcaa254)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:19.312980 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ac5f3e-1c98-470a-89d7-253284364e98 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:19.314899 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 104/205] 10.222.0.99 () {56 vars in 1177 bytes} [Thu Jul 14 15:22:19 2022] GET /identity/v3/users/c83c274d5da64fed90c95ef77dcaa254 => generated 289 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:19.320756 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23810169-9ee8-42fa-ad8a-4d1e090e9999 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:19.488222 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23810169-9ee8-42fa-ad8a-4d1e090e9999 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:19.529499 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4433a4-8ab2-46b2-8067-5f98730331c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:19.530508 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4433a4-8ab2-46b2-8067-5f98730331c0 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:19.530997 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4433a4-8ab2-46b2-8067-5f98730331c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:19.531717 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4433a4-8ab2-46b2-8067-5f98730331c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:19.552352 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:19.607241 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4433a4-8ab2-46b2-8067-5f98730331c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:19.609229 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4433a4-8ab2-46b2-8067-5f98730331c0 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:19.761947 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 102/206] 10.222.0.99 () {58 vars in 1282 bytes} [Thu Jul 14 15:22:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 15:22:19.767636 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ac5f3e-1c98-470a-89d7-253284364e98 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:19.858242 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:19.925758 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ac5f3e-1c98-470a-89d7-253284364e98 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:19.980435 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401582a9-75bb-4c3b-9085-62884e2567f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:19.981669 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401582a9-75bb-4c3b-9085-62884e2567f9 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:19.982160 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401582a9-75bb-4c3b-9085-62884e2567f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:19.982665 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401582a9-75bb-4c3b-9085-62884e2567f9 None admin] PATH_INFO: `/v3/projects/1d9e82a46b1c4b25939e533c26cb44df` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:19.986152 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401582a9-75bb-4c3b-9085-62884e2567f9 None admin] RBAC: Authorizing `identity:get_project(project_id=1d9e82a46b1c4b25939e533c26cb44df)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:19.987880 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401582a9-75bb-4c3b-9085-62884e2567f9 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:19.989918 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 105/207] 10.222.0.99 () {56 vars in 1186 bytes} [Thu Jul 14 15:22:19 2022] GET /identity/v3/projects/1d9e82a46b1c4b25939e533c26cb44df => generated 298 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:19.999621 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4433a4-8ab2-46b2-8067-5f98730331c0 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:20.080255 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:20.159887 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4433a4-8ab2-46b2-8067-5f98730331c0 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:20.203292 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04ede0e-ba32-4457-9504-5fa3297619e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:20.204002 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04ede0e-ba32-4457-9504-5fa3297619e4 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:20.204293 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04ede0e-ba32-4457-9504-5fa3297619e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:20.204578 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04ede0e-ba32-4457-9504-5fa3297619e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:20.205438 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04ede0e-ba32-4457-9504-5fa3297619e4 None admin] RBAC: Adding query filter params (role.id=cc5c3091b148423684e5e966150bc6c6, scope.project.id=1d9e82a46b1c4b25939e533c26cb44df, user.id=c83c274d5da64fed90c95ef77dcaa254) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:22:20.205839 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04ede0e-ba32-4457-9504-5fa3297619e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:20.212708 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04ede0e-ba32-4457-9504-5fa3297619e4 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:20.219520 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 103/208] 10.222.0.99 () {56 vars in 1374 bytes} [Thu Jul 14 15:22:19 2022] GET /identity/v3/role_assignments?user.id=c83c274d5da64fed90c95ef77dcaa254&scope.project.id=1d9e82a46b1c4b25939e533c26cb44df&role.id=cc5c3091b148423684e5e966150bc6c6 => generated 608 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:20.231562 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed0227f-78e5-44ae-b096-3805847bbee6 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:20.231933 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed0227f-78e5-44ae-b096-3805847bbee6 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:20.232222 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed0227f-78e5-44ae-b096-3805847bbee6 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:20.233676 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 106/209] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:22:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:22:20.239543 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30d9154-8d1f-4e87-bcb0-e0255cd050c2 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:20.239915 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30d9154-8d1f-4e87-bcb0-e0255cd050c2 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:20.240159 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30d9154-8d1f-4e87-bcb0-e0255cd050c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:20.268412 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-d30d9154-8d1f-4e87-bcb0-e0255cd050c2 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:22:20.269655 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d30d9154-8d1f-4e87-bcb0-e0255cd050c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:22:20.378588 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 104/210] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:22:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:22:20.410731 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed0227f-78e5-44ae-b096-3805847bbee6 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:20.535961 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed0227f-78e5-44ae-b096-3805847bbee6 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:20.554606 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:20.575572 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5bec8a-8215-44a0-a073-b0ed217e19e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:20.576416 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5bec8a-8215-44a0-a073-b0ed217e19e7 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:20.576703 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5bec8a-8215-44a0-a073-b0ed217e19e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:20.576982 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5bec8a-8215-44a0-a073-b0ed217e19e7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:20.590001 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5bec8a-8215-44a0-a073-b0ed217e19e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:22:20.597964 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-1e5bec8a-8215-44a0-a073-b0ed217e19e7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 15:22:20.597964 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:22:20.597964 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:22:20.597964 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:22:20.597964 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:22:20.597964 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:22:20.597964 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:22:20.597964 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:22:20.597964 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:22:20.597964 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:22:20.597964 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:22:20.597964 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:22:20.597964 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:22:20.597964 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:22:20.597964 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:22:20.597964 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:22:20.598839 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:22:20.598839 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:22:20.598839 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:22:20.598839 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:22:20.598839 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:22:20.598839 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:22:20.598839 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:22:20.598839 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:22:20.598839 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:22:20.598839 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:22:20.598839 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:22:20.598839 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:22:20.598839 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:22:20.598839 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:22:20.598839 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:22:20.598839 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:22:20.599639 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:22:20.599639 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:22:20.599639 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:22:20.599639 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:22:20.599639 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:22:20.599639 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:22:20.599639 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:22:20.599639 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:22:20.599639 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:22:20.599639 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:22:20.599639 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 15:22:20.599639 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:22:20.599639 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 107/211] 10.222.0.99 () {56 vars in 1099 bytes} [Thu Jul 14 15:22:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:22:20.606340 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30d9154-8d1f-4e87-bcb0-e0255cd050c2 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:20.619267 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7c664c-83bf-4daf-b4a7-8ff0bc97f958 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:20.619617 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7c664c-83bf-4daf-b4a7-8ff0bc97f958 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:20.619901 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7c664c-83bf-4daf-b4a7-8ff0bc97f958 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:20.621453 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 108/212] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:22:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 15:22:20.626580 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0721a42-2da2-44e1-961a-eb4eeae8ae4f None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:20.627107 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0721a42-2da2-44e1-961a-eb4eeae8ae4f None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:20.627393 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0721a42-2da2-44e1-961a-eb4eeae8ae4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:20.651170 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-a0721a42-2da2-44e1-961a-eb4eeae8ae4f None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:22:20.651170 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-a0721a42-2da2-44e1-961a-eb4eeae8ae4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:22:20.661495 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:20.727569 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 109/213] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:22:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 15:22:20.729601 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30d9154-8d1f-4e87-bcb0-e0255cd050c2 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:20.729823 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e5d6e8-74af-42d5-b727-ca5e0396667d None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:20.730123 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e5d6e8-74af-42d5-b727-ca5e0396667d None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:20.730412 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e5d6e8-74af-42d5-b727-ca5e0396667d None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:20.732013 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 110/214] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:22:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:22:20.739951 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867795f3-842f-4ef9-91d2-2af7f1f322a1 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:20.740250 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867795f3-842f-4ef9-91d2-2af7f1f322a1 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:20.740533 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867795f3-842f-4ef9-91d2-2af7f1f322a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:20.758623 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-867795f3-842f-4ef9-91d2-2af7f1f322a1 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:22:20.760286 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-867795f3-842f-4ef9-91d2-2af7f1f322a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:22:20.772237 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e39ee2-2ea3-4e08-b305-6274013d6c6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:20.772923 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e39ee2-2ea3-4e08-b305-6274013d6c6c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:20.773211 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e39ee2-2ea3-4e08-b305-6274013d6c6c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:20.773492 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e39ee2-2ea3-4e08-b305-6274013d6c6c None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:20.774256 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e39ee2-2ea3-4e08-b305-6274013d6c6c None admin] RBAC: Adding query filter params (name=member) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:22:20.774569 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e39ee2-2ea3-4e08-b305-6274013d6c6c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:20.776749 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e39ee2-2ea3-4e08-b305-6274013d6c6c None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:20.785847 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 105/215] 10.222.0.99 () {56 vars in 1101 bytes} [Thu Jul 14 15:22:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:20.790471 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e39ee2-2ea3-4e08-b305-6274013d6c6c None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:20.840204 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 111/216] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:22:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 15:22:20.841210 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867795f3-842f-4ef9-91d2-2af7f1f322a1 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:20.890594 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:20.900773 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:20.904500 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e39ee2-2ea3-4e08-b305-6274013d6c6c None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:20.940858 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ff8412-ac42-4e6d-8de8-473890cdba85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:20.941661 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ff8412-ac42-4e6d-8de8-473890cdba85 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:20.941943 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ff8412-ac42-4e6d-8de8-473890cdba85 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:20.942219 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ff8412-ac42-4e6d-8de8-473890cdba85 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:20.943394 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ff8412-ac42-4e6d-8de8-473890cdba85 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:20.944877 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ff8412-ac42-4e6d-8de8-473890cdba85 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:20.946498 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 106/217] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:22:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:20.952744 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ff8412-ac42-4e6d-8de8-473890cdba85 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:20.977547 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867795f3-842f-4ef9-91d2-2af7f1f322a1 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:21.019894 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a06e404-56ea-46cb-9225-5386faab9607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:21.021164 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a06e404-56ea-46cb-9225-5386faab9607 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:21.021656 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a06e404-56ea-46cb-9225-5386faab9607 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:21.022506 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a06e404-56ea-46cb-9225-5386faab9607 None admin] PATH_INFO: `/v3/users/3f2dd8e31dd1411281812a0144ac8816` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:21.039131 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a06e404-56ea-46cb-9225-5386faab9607 None admin] RBAC: Authorizing `identity:get_user(user_id=3f2dd8e31dd1411281812a0144ac8816)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:21.040869 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a06e404-56ea-46cb-9225-5386faab9607 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:21.042789 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 112/218] 10.222.0.99 () {56 vars in 1177 bytes} [Thu Jul 14 15:22:20 2022] GET /identity/v3/users/3f2dd8e31dd1411281812a0144ac8816 => generated 297 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:21.043761 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a06e404-56ea-46cb-9225-5386faab9607 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:21.073980 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ff8412-ac42-4e6d-8de8-473890cdba85 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:21.109704 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db6c27e-ef6a-489f-b884-a3be2eaffcad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:21.110492 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db6c27e-ef6a-489f-b884-a3be2eaffcad None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:21.110780 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db6c27e-ef6a-489f-b884-a3be2eaffcad None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:21.111059 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db6c27e-ef6a-489f-b884-a3be2eaffcad None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:21.122379 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db6c27e-ef6a-489f-b884-a3be2eaffcad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:22:21.130411 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-5db6c27e-ef6a-489f-b884-a3be2eaffcad None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:22:21.130411 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:22:21.130411 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:22:21.130411 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:22:21.130411 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:22:21.130411 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:22:21.130411 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:22:21.130411 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:22:21.130411 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:22:21.130411 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:22:21.130411 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:22:21.130411 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:22:21.130411 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:22:21.130411 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:22:21.130411 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:22:21.130411 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:22:21.131347 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:22:21.131347 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:22:21.131347 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:22:21.131347 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:22:21.131347 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:22:21.131347 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:22:21.131347 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:22:21.131347 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:22:21.131347 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:22:21.131347 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:22:21.131347 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:22:21.131347 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:22:21.131347 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:22:21.131347 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:22:21.131347 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:22:21.131347 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:22:21.132246 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:22:21.132246 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:22:21.132246 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:22:21.132246 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:22:21.132246 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:22:21.132246 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:22:21.132246 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:22:21.132246 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:22:21.132246 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:22:21.132246 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:22:21.132246 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:22:21.132246 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:22:21.138145 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 107/219] 10.222.0.99 () {56 vars in 1096 bytes} [Thu Jul 14 15:22:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:22:21.139313 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db6c27e-ef6a-489f-b884-a3be2eaffcad None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:21.158608 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:21.167435 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a06e404-56ea-46cb-9225-5386faab9607 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:21.172889 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:21.203495 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951533ce-1ecc-4d00-a246-9f81ac585fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:21.204123 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951533ce-1ecc-4d00-a246-9f81ac585fe6 None admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:21.204411 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951533ce-1ecc-4d00-a246-9f81ac585fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:21.204692 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951533ce-1ecc-4d00-a246-9f81ac585fe6 None admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:21.205407 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951533ce-1ecc-4d00-a246-9f81ac585fe6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:21.206859 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951533ce-1ecc-4d00-a246-9f81ac585fe6 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:21.225700 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 113/220] 10.222.0.99 () {60 vars in 1132 bytes} [Thu Jul 14 15:22:21 2022] POST /identity/v3/users => generated 297 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:22:21.233566 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951533ce-1ecc-4d00-a246-9f81ac585fe6 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:21.267556 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db6c27e-ef6a-489f-b884-a3be2eaffcad None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:21.314368 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e89746-603c-4517-ae3f-26e5da95e3d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:21.314624 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e89746-603c-4517-ae3f-26e5da95e3d8 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:21.314713 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e89746-603c-4517-ae3f-26e5da95e3d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:21.314987 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e89746-603c-4517-ae3f-26e5da95e3d8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:21.344703 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951533ce-1ecc-4d00-a246-9f81ac585fe6 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:21.385350 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acf944b-3879-4ec6-99bb-9bd19d2fb141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:21.386124 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acf944b-3879-4ec6-99bb-9bd19d2fb141 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:21.386407 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acf944b-3879-4ec6-99bb-9bd19d2fb141 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:21.386683 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acf944b-3879-4ec6-99bb-9bd19d2fb141 None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:21.387452 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acf944b-3879-4ec6-99bb-9bd19d2fb141 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:22:21.387788 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acf944b-3879-4ec6-99bb-9bd19d2fb141 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:21.389204 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acf944b-3879-4ec6-99bb-9bd19d2fb141 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:21.394174 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e89746-603c-4517-ae3f-26e5da95e3d8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:21.396002 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e89746-603c-4517-ae3f-26e5da95e3d8 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:21.398109 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 114/221] 10.222.0.99 () {56 vars in 1099 bytes} [Thu Jul 14 15:22:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:21.398109 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfe9111-0d55-4992-9cd5-211aa5f0f50d None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:21.398109 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfe9111-0d55-4992-9cd5-211aa5f0f50d None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:21.400787 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfe9111-0d55-4992-9cd5-211aa5f0f50d None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:21.403052 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 115/222] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:22:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:22:21.406114 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfe9111-0d55-4992-9cd5-211aa5f0f50d None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:21.424355 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:21.443969 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:21.546268 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfe9111-0d55-4992-9cd5-211aa5f0f50d None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:21.564669 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 108/223] 10.222.0.99 () {58 vars in 1282 bytes} [Thu Jul 14 15:22:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 15:22:21.565222 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72930bda-c99a-4cca-861f-0baa0ca009c9 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:21.565512 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72930bda-c99a-4cca-861f-0baa0ca009c9 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:21.565813 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72930bda-c99a-4cca-861f-0baa0ca009c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:21.591410 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-72930bda-c99a-4cca-861f-0baa0ca009c9 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:22:21.591410 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-72930bda-c99a-4cca-861f-0baa0ca009c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:22:21.615970 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf173339-21dc-46ba-8bc5-4b4a3e402b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:21.617970 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf173339-21dc-46ba-8bc5-4b4a3e402b8d None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:21.618304 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf173339-21dc-46ba-8bc5-4b4a3e402b8d None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:21.618577 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf173339-21dc-46ba-8bc5-4b4a3e402b8d None admin] PATH_INFO: `/v3/users/6ddb9f84c97f418a88ac32e44ba83caa` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:21.620708 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf173339-21dc-46ba-8bc5-4b4a3e402b8d None admin] RBAC: Authorizing `identity:get_user(user_id=6ddb9f84c97f418a88ac32e44ba83caa)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:21.622339 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf173339-21dc-46ba-8bc5-4b4a3e402b8d None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:21.624163 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 116/224] 10.222.0.99 () {56 vars in 1177 bytes} [Thu Jul 14 15:22:21 2022] GET /identity/v3/users/6ddb9f84c97f418a88ac32e44ba83caa => generated 283 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:21.624826 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf173339-21dc-46ba-8bc5-4b4a3e402b8d None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:21.676276 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 109/225] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:22:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:22:21.677813 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72930bda-c99a-4cca-861f-0baa0ca009c9 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:21.720259 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:21.747224 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf173339-21dc-46ba-8bc5-4b4a3e402b8d None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:21.748994 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:21.798306 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef0af88-71cd-4c8c-8e5f-7c9f835c7319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:21.799489 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef0af88-71cd-4c8c-8e5f-7c9f835c7319 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:21.799762 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72930bda-c99a-4cca-861f-0baa0ca009c9 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:21.800190 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef0af88-71cd-4c8c-8e5f-7c9f835c7319 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:21.800643 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef0af88-71cd-4c8c-8e5f-7c9f835c7319 None admin] PATH_INFO: `/v3/projects/925bf9baf40d4a888a9d633d1deb5134` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:21.803214 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef0af88-71cd-4c8c-8e5f-7c9f835c7319 None admin] RBAC: Authorizing `identity:get_project(project_id=925bf9baf40d4a888a9d633d1deb5134)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:21.804909 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef0af88-71cd-4c8c-8e5f-7c9f835c7319 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:21.806987 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 117/226] 10.222.0.99 () {56 vars in 1186 bytes} [Thu Jul 14 15:22:21 2022] GET /identity/v3/projects/925bf9baf40d4a888a9d633d1deb5134 => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:21.808026 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef0af88-71cd-4c8c-8e5f-7c9f835c7319 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:21.840643 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696d3f95-f609-4bdf-835f-3fe75fb42cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:21.841687 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696d3f95-f609-4bdf-835f-3fe75fb42cad None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:21.842048 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696d3f95-f609-4bdf-835f-3fe75fb42cad None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:21.842405 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696d3f95-f609-4bdf-835f-3fe75fb42cad None admin] PATH_INFO: `/v3/projects/925bf9baf40d4a888a9d633d1deb5134` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:21.843876 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696d3f95-f609-4bdf-835f-3fe75fb42cad None admin] RBAC: Authorizing `identity:get_project(project_id=925bf9baf40d4a888a9d633d1deb5134)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:21.845532 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696d3f95-f609-4bdf-835f-3fe75fb42cad None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:21.847502 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 110/227] 10.222.0.99 () {56 vars in 1186 bytes} [Thu Jul 14 15:22:21 2022] GET /identity/v3/projects/925bf9baf40d4a888a9d633d1deb5134 => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:21.848777 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696d3f95-f609-4bdf-835f-3fe75fb42cad None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:21.929120 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef0af88-71cd-4c8c-8e5f-7c9f835c7319 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:21.961765 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696d3f95-f609-4bdf-835f-3fe75fb42cad None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:21.966862 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecf3500-1186-4435-ac33-aa79ffe14e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:21.967971 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecf3500-1186-4435-ac33-aa79ffe14e82 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:21.968436 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecf3500-1186-4435-ac33-aa79ffe14e82 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:21.968865 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecf3500-1186-4435-ac33-aa79ffe14e82 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:21.981331 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecf3500-1186-4435-ac33-aa79ffe14e82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:22:21.996371 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-eecf3500-1186-4435-ac33-aa79ffe14e82 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 15:22:21.996371 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:22:21.996371 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:22:21.996371 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:22:21.996371 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:22:21.996371 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:22:21.996371 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:22:21.996371 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:22:21.996371 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:22:21.996371 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:22:21.996371 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:22:21.996371 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:22:21.996371 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:22:21.996371 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:22:21.996371 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:22:21.996371 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:22:21.997234 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:22:21.997234 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:22:21.997234 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:22:21.997234 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:22:21.997234 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:22:21.997234 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:22:21.997234 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:22:21.997234 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:22:21.997234 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:22:21.997234 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:22:21.997234 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:22:21.997234 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:22:21.997234 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:22:21.997234 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:22:21.997234 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:22:21.997234 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:22:21.998040 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:22:21.998040 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:22:21.998040 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:22:21.998040 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:22:21.998040 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:22:21.998040 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:22:21.998040 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:22:21.998040 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:22:21.998040 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:22:21.998040 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:22:21.998040 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 15:22:21.998040 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:22:21.998040 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 118/228] 10.222.0.99 () {56 vars in 1114 bytes} [Thu Jul 14 15:22:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:22:21.998840 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecf3500-1186-4435-ac33-aa79ffe14e82 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:22.011642 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:22.016588 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68163011-6996-4b72-ada2-adf95e7e4278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:22.017667 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68163011-6996-4b72-ada2-adf95e7e4278 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:22.018556 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68163011-6996-4b72-ada2-adf95e7e4278 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:22.018980 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68163011-6996-4b72-ada2-adf95e7e4278 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:22.020295 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68163011-6996-4b72-ada2-adf95e7e4278 None admin] RBAC: Adding query filter params (role.id=b8c98fddc0fd4436b577fd1d3361bc38, scope.project.id=925bf9baf40d4a888a9d633d1deb5134, user.id=3f2dd8e31dd1411281812a0144ac8816) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:22:22.020999 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68163011-6996-4b72-ada2-adf95e7e4278 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:22.022964 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68163011-6996-4b72-ada2-adf95e7e4278 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:22.031527 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:22.032171 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 111/229] 10.222.0.99 () {56 vars in 1374 bytes} [Thu Jul 14 15:22:21 2022] GET /identity/v3/role_assignments?user.id=3f2dd8e31dd1411281812a0144ac8816&scope.project.id=925bf9baf40d4a888a9d633d1deb5134&role.id=b8c98fddc0fd4436b577fd1d3361bc38 => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:22.041789 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68163011-6996-4b72-ada2-adf95e7e4278 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:22.161315 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecf3500-1186-4435-ac33-aa79ffe14e82 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:22.210024 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3928b2a1-2e60-4eed-9e74-047620d11859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:22.210937 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3928b2a1-2e60-4eed-9e74-047620d11859 None admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:22.211234 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3928b2a1-2e60-4eed-9e74-047620d11859 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:22.211516 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3928b2a1-2e60-4eed-9e74-047620d11859 None admin] PATH_INFO: `/v3/projects/925bf9baf40d4a888a9d633d1deb5134/users/6ddb9f84c97f418a88ac32e44ba83caa/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:22.214332 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3928b2a1-2e60-4eed-9e74-047620d11859 None admin] RBAC: Authorizing `identity:create_grant(project_id=925bf9baf40d4a888a9d633d1deb5134, user_id=6ddb9f84c97f418a88ac32e44ba83caa, role_id=d0f6ba1314f14d9f9c816bdb5d0c3aac)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:22.215917 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3928b2a1-2e60-4eed-9e74-047620d11859 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:22.235771 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 119/230] 10.222.0.99 () {58 vars in 1439 bytes} [Thu Jul 14 15:22:21 2022] PUT /identity/v3/projects/925bf9baf40d4a888a9d633d1deb5134/users/6ddb9f84c97f418a88ac32e44ba83caa/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:22:22.255149 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68163011-6996-4b72-ada2-adf95e7e4278 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:22.300433 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a139bd-44d2-420c-8d03-17147f27ab73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:22.301142 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a139bd-44d2-420c-8d03-17147f27ab73 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:22.301435 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a139bd-44d2-420c-8d03-17147f27ab73 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:22.301724 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a139bd-44d2-420c-8d03-17147f27ab73 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:22.302514 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a139bd-44d2-420c-8d03-17147f27ab73 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:22:22.302860 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a139bd-44d2-420c-8d03-17147f27ab73 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:22.304621 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a139bd-44d2-420c-8d03-17147f27ab73 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:22.320258 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 112/231] 10.222.0.99 () {56 vars in 1111 bytes} [Thu Jul 14 15:22:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:22.326602 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3928b2a1-2e60-4eed-9e74-047620d11859 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:22.419320 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:22.456821 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3928b2a1-2e60-4eed-9e74-047620d11859 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:22.496009 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c40e195-d892-4dd4-82ca-544d9a3fb93c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:22.496931 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c40e195-d892-4dd4-82ca-544d9a3fb93c None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:22.497229 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c40e195-d892-4dd4-82ca-544d9a3fb93c None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:22.497514 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c40e195-d892-4dd4-82ca-544d9a3fb93c None admin] PATH_INFO: `/v3/users/6ddb9f84c97f418a88ac32e44ba83caa` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:22.498806 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c40e195-d892-4dd4-82ca-544d9a3fb93c None admin] RBAC: Authorizing `identity:get_user(user_id=6ddb9f84c97f418a88ac32e44ba83caa)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:22.500394 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c40e195-d892-4dd4-82ca-544d9a3fb93c None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:22.502673 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 120/232] 10.222.0.99 () {56 vars in 1177 bytes} [Thu Jul 14 15:22:22 2022] GET /identity/v3/users/6ddb9f84c97f418a88ac32e44ba83caa => generated 283 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:22.508421 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a139bd-44d2-420c-8d03-17147f27ab73 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:22.568610 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:22.625044 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a139bd-44d2-420c-8d03-17147f27ab73 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:22.668819 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439a044b-4e21-4f91-8923-ae0ef044654b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:22.669786 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439a044b-4e21-4f91-8923-ae0ef044654b None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:22.670022 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439a044b-4e21-4f91-8923-ae0ef044654b None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:22.670311 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439a044b-4e21-4f91-8923-ae0ef044654b None admin] PATH_INFO: `/v3/projects/925bf9baf40d4a888a9d633d1deb5134` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:22.671563 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439a044b-4e21-4f91-8923-ae0ef044654b None admin] RBAC: Authorizing `identity:get_project(project_id=925bf9baf40d4a888a9d633d1deb5134)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:22.673123 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439a044b-4e21-4f91-8923-ae0ef044654b None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:22.675200 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 113/233] 10.222.0.99 () {56 vars in 1186 bytes} [Thu Jul 14 15:22:22 2022] GET /identity/v3/projects/925bf9baf40d4a888a9d633d1deb5134 => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:22.681140 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c40e195-d892-4dd4-82ca-544d9a3fb93c None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:22.838231 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c40e195-d892-4dd4-82ca-544d9a3fb93c None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:22.886774 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba6e00c-bea4-4cdf-96c2-1628be85e396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:22.888063 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba6e00c-bea4-4cdf-96c2-1628be85e396 None admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:22.888545 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba6e00c-bea4-4cdf-96c2-1628be85e396 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:22.889039 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba6e00c-bea4-4cdf-96c2-1628be85e396 None admin] PATH_INFO: `/v3/projects/925bf9baf40d4a888a9d633d1deb5134/users/6ddb9f84c97f418a88ac32e44ba83caa/roles/3627b4f45c8c4f41a7def6b15f1b8225` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:22.891584 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba6e00c-bea4-4cdf-96c2-1628be85e396 None admin] RBAC: Authorizing `identity:create_grant(project_id=925bf9baf40d4a888a9d633d1deb5134, user_id=6ddb9f84c97f418a88ac32e44ba83caa, role_id=3627b4f45c8c4f41a7def6b15f1b8225)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:22.893289 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba6e00c-bea4-4cdf-96c2-1628be85e396 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:22.896656 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:22.908508 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 121/234] 10.222.0.99 () {58 vars in 1439 bytes} [Thu Jul 14 15:22:22 2022] PUT /identity/v3/projects/925bf9baf40d4a888a9d633d1deb5134/users/6ddb9f84c97f418a88ac32e44ba83caa/roles/3627b4f45c8c4f41a7def6b15f1b8225 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:22:23.632788 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b20c40-72cc-49c7-a918-d58574436052 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:23.633796 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b20c40-72cc-49c7-a918-d58574436052 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:23.634286 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b20c40-72cc-49c7-a918-d58574436052 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:23.636804 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 114/235] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:22:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:22:23.642419 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fff68b-e365-4039-8328-75419f56854a None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:23.643347 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fff68b-e365-4039-8328-75419f56854a None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:23.643879 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fff68b-e365-4039-8328-75419f56854a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:23.661839 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-44fff68b-e365-4039-8328-75419f56854a None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:22:23.663566 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-44fff68b-e365-4039-8328-75419f56854a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:22:23.739230 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 122/236] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:22:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:22:23.764617 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b20c40-72cc-49c7-a918-d58574436052 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:23.880274 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b20c40-72cc-49c7-a918-d58574436052 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:23.883751 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22046b65-f678-4061-805a-48a6442ddfa1 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:23.884265 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22046b65-f678-4061-805a-48a6442ddfa1 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:23.884796 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22046b65-f678-4061-805a-48a6442ddfa1 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:23.886793 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 123/237] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:22:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:22:23.891935 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4f5adc-b085-41e1-9cf6-ee95fe77d1de None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:23.892459 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4f5adc-b085-41e1-9cf6-ee95fe77d1de None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:23.892931 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4f5adc-b085-41e1-9cf6-ee95fe77d1de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:23.912757 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-1c4f5adc-b085-41e1-9cf6-ee95fe77d1de None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:22:23.914347 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-1c4f5adc-b085-41e1-9cf6-ee95fe77d1de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:22:23.929391 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:23.959181 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd81e4c-0978-4fea-942e-f6e6c2209666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:23.960030 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd81e4c-0978-4fea-942e-f6e6c2209666 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:23.960321 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd81e4c-0978-4fea-942e-f6e6c2209666 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:23.960603 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd81e4c-0978-4fea-942e-f6e6c2209666 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:23.981461 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd81e4c-0978-4fea-942e-f6e6c2209666 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:22:23.993375 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 124/238] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:22:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:22:23.997649 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-5dd81e4c-0978-4fea-942e-f6e6c2209666 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 15:22:23.997649 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:22:23.997649 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:22:23.997649 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:22:23.997649 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:22:23.997649 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:22:23.997649 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:22:23.997649 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:22:23.997649 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:22:23.997649 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:22:23.997649 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:22:23.997649 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:22:23.997649 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:22:23.997649 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:22:23.997649 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:22:23.997649 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:22:23.998523 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:22:23.998523 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:22:23.998523 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:22:23.998523 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:22:23.998523 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:22:23.998523 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:22:23.998523 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:22:23.998523 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:22:23.998523 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:22:23.998523 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:22:23.998523 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:22:23.998523 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:22:23.998523 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:22:23.998523 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:22:23.998523 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:22:23.998523 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:22:23.999289 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:22:23.999289 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:22:23.999289 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:22:23.999289 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:22:23.999289 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:22:23.999289 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:22:23.999289 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:22:23.999289 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:22:23.999289 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:22:23.999289 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:22:23.999289 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 15:22:23.999289 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:22:23.999289 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 115/239] 10.222.0.99 () {56 vars in 1099 bytes} [Thu Jul 14 15:22:23 2022] GET /identity/v3/roles/reader => generated 84 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:22:24.006474 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4f5adc-b085-41e1-9cf6-ee95fe77d1de None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:24.030555 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd81e4c-0978-4fea-942e-f6e6c2209666 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:24.073625 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:24.125823 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4f5adc-b085-41e1-9cf6-ee95fe77d1de None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:24.163183 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d312e005-4afe-4c5f-a9ca-ccf983cc9012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:24.164268 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d312e005-4afe-4c5f-a9ca-ccf983cc9012 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:24.164750 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d312e005-4afe-4c5f-a9ca-ccf983cc9012 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:24.165288 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d312e005-4afe-4c5f-a9ca-ccf983cc9012 None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:24.166345 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d312e005-4afe-4c5f-a9ca-ccf983cc9012 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:22:24.167154 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d312e005-4afe-4c5f-a9ca-ccf983cc9012 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:24.168859 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d312e005-4afe-4c5f-a9ca-ccf983cc9012 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:24.178505 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 125/240] 10.222.0.99 () {56 vars in 1101 bytes} [Thu Jul 14 15:22:24 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:24.184981 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d312e005-4afe-4c5f-a9ca-ccf983cc9012 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:24.292605 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d312e005-4afe-4c5f-a9ca-ccf983cc9012 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:24.303523 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd81e4c-0978-4fea-942e-f6e6c2209666 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:24.311350 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:24.332637 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28208e3-93c9-4d46-b07a-53a44de9d14f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:24.333893 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28208e3-93c9-4d46-b07a-53a44de9d14f None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:24.334399 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28208e3-93c9-4d46-b07a-53a44de9d14f None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:24.334960 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28208e3-93c9-4d46-b07a-53a44de9d14f None admin] PATH_INFO: `/v3/users/c54065f8d99f443a8e0550a404b05514` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:24.350391 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28208e3-93c9-4d46-b07a-53a44de9d14f None admin] RBAC: Authorizing `identity:get_user(user_id=c54065f8d99f443a8e0550a404b05514)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:24.352155 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28208e3-93c9-4d46-b07a-53a44de9d14f None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:24.355330 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 126/241] 10.222.0.99 () {56 vars in 1177 bytes} [Thu Jul 14 15:22:24 2022] GET /identity/v3/users/c54065f8d99f443a8e0550a404b05514 => generated 297 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:22:24.360801 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28208e3-93c9-4d46-b07a-53a44de9d14f None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:24.408358 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac30c862-ac20-4b83-a453-960b3e1c8eae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:24.409189 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac30c862-ac20-4b83-a453-960b3e1c8eae None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:24.409483 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac30c862-ac20-4b83-a453-960b3e1c8eae None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:24.409768 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac30c862-ac20-4b83-a453-960b3e1c8eae None admin] PATH_INFO: `/v3/domains/default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:24.411030 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac30c862-ac20-4b83-a453-960b3e1c8eae None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:24.412541 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac30c862-ac20-4b83-a453-960b3e1c8eae None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:24.414416 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 116/242] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:22:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 384 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:24.420758 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac30c862-ac20-4b83-a453-960b3e1c8eae None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:24.474236 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28208e3-93c9-4d46-b07a-53a44de9d14f None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:24.512791 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3431919-d34b-470f-aa8b-3a2e4e2e14f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:24.514309 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3431919-d34b-470f-aa8b-3a2e4e2e14f6 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:24.515012 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3431919-d34b-470f-aa8b-3a2e4e2e14f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:24.515725 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3431919-d34b-470f-aa8b-3a2e4e2e14f6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:24.533998 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:24.564517 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:24.591007 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3431919-d34b-470f-aa8b-3a2e4e2e14f6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:24.592526 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3431919-d34b-470f-aa8b-3a2e4e2e14f6 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:24.640641 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac30c862-ac20-4b83-a453-960b3e1c8eae None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:24.689789 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0844a8a9-3f5d-47c1-881f-e90f81c90b33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:24.690395 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0844a8a9-3f5d-47c1-881f-e90f81c90b33 None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:24.690714 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0844a8a9-3f5d-47c1-881f-e90f81c90b33 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:24.690954 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0844a8a9-3f5d-47c1-881f-e90f81c90b33 None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:24.691711 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0844a8a9-3f5d-47c1-881f-e90f81c90b33 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:24.693185 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0844a8a9-3f5d-47c1-881f-e90f81c90b33 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:24.731029 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 117/243] 10.222.0.99 () {60 vars in 1132 bytes} [Thu Jul 14 15:22:24 2022] POST /identity/v3/users => generated 293 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:22:24.746913 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0461eba-2be3-4fd9-9a9a-c7d711cb9ba8 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:24.747144 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0461eba-2be3-4fd9-9a9a-c7d711cb9ba8 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:24.747435 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0461eba-2be3-4fd9-9a9a-c7d711cb9ba8 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:24.749036 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 118/244] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:22:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 15:22:24.754918 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47a8563-2af1-4ff5-9cd4-bc538d6ba9f3 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:24.755202 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47a8563-2af1-4ff5-9cd4-bc538d6ba9f3 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:24.755488 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47a8563-2af1-4ff5-9cd4-bc538d6ba9f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:24.758035 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 127/245] 10.222.0.99 () {58 vars in 1282 bytes} [Thu Jul 14 15:22:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 15:22:24.764477 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3431919-d34b-470f-aa8b-3a2e4e2e14f6 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:24.793307 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-e47a8563-2af1-4ff5-9cd4-bc538d6ba9f3 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:22:24.794733 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e47a8563-2af1-4ff5-9cd4-bc538d6ba9f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:22:24.868283 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:24.923755 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3431919-d34b-470f-aa8b-3a2e4e2e14f6 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:24.951528 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 119/246] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:22:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 15:22:24.968679 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07fa71b-72b9-4373-a1f0-7e9f93b53713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:24.969880 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07fa71b-72b9-4373-a1f0-7e9f93b53713 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:24.970370 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07fa71b-72b9-4373-a1f0-7e9f93b53713 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:24.970881 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07fa71b-72b9-4373-a1f0-7e9f93b53713 None admin] PATH_INFO: `/v3/projects/925bf9baf40d4a888a9d633d1deb5134` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:24.972341 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07fa71b-72b9-4373-a1f0-7e9f93b53713 None admin] RBAC: Authorizing `identity:get_project(project_id=925bf9baf40d4a888a9d633d1deb5134)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:24.974003 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07fa71b-72b9-4373-a1f0-7e9f93b53713 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:24.975927 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47a8563-2af1-4ff5-9cd4-bc538d6ba9f3 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:24.977084 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 128/247] 10.222.0.99 () {56 vars in 1186 bytes} [Thu Jul 14 15:22:24 2022] GET /identity/v3/projects/925bf9baf40d4a888a9d633d1deb5134 => generated 302 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:24.983295 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07fa71b-72b9-4373-a1f0-7e9f93b53713 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:25.020909 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:25.106143 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47a8563-2af1-4ff5-9cd4-bc538d6ba9f3 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:25.106440 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07fa71b-72b9-4373-a1f0-7e9f93b53713 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:25.144704 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5a112d-e1a5-4504-916b-8da235a85f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:25.145350 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ad60c4-7944-4641-a6c7-12895664f934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:25.145694 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5a112d-e1a5-4504-916b-8da235a85f35 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:25.146022 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ad60c4-7944-4641-a6c7-12895664f934 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:25.146124 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5a112d-e1a5-4504-916b-8da235a85f35 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:25.146314 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ad60c4-7944-4641-a6c7-12895664f934 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:25.146514 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5a112d-e1a5-4504-916b-8da235a85f35 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:25.146622 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ad60c4-7944-4641-a6c7-12895664f934 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:25.147438 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ad60c4-7944-4641-a6c7-12895664f934 None admin] RBAC: Adding query filter params (role.id=cc5c3091b148423684e5e966150bc6c6, scope.project.id=925bf9baf40d4a888a9d633d1deb5134, user.id=c54065f8d99f443a8e0550a404b05514) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:22:25.147789 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ad60c4-7944-4641-a6c7-12895664f934 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:25.149377 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ad60c4-7944-4641-a6c7-12895664f934 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:25.156046 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 129/248] 10.222.0.99 () {56 vars in 1374 bytes} [Thu Jul 14 15:22:24 2022] GET /identity/v3/role_assignments?user.id=c54065f8d99f443a8e0550a404b05514&scope.project.id=925bf9baf40d4a888a9d633d1deb5134&role.id=cc5c3091b148423684e5e966150bc6c6 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:25.159524 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5a112d-e1a5-4504-916b-8da235a85f35 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:22:25.167936 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-5f5a112d-e1a5-4504-916b-8da235a85f35 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 15:22:25.167936 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:22:25.167936 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:22:25.167936 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:22:25.167936 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:22:25.167936 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:22:25.167936 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:22:25.167936 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:22:25.167936 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:22:25.167936 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:22:25.167936 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:22:25.167936 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:22:25.167936 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:22:25.167936 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:22:25.167936 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:22:25.167936 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:22:25.169641 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:22:25.169641 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:22:25.169641 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:22:25.169641 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:22:25.169641 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:22:25.169641 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:22:25.169641 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:22:25.169641 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:22:25.169641 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:22:25.169641 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:22:25.169641 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:22:25.169641 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:22:25.169641 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:22:25.169641 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:22:25.169641 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:22:25.169641 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:22:25.171267 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:22:25.171267 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:22:25.171267 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:22:25.171267 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:22:25.171267 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:22:25.171267 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:22:25.171267 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:22:25.171267 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:22:25.171267 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:22:25.171267 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:22:25.171267 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 15:22:25.171267 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:22:25.172475 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 120/249] 10.222.0.99 () {56 vars in 1099 bytes} [Thu Jul 14 15:22:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:22:25.176388 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ad60c4-7944-4641-a6c7-12895664f934 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:25.191642 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:25.201474 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf9c8ec-b653-4c50-8daa-fafa961caf89 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:25.201823 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf9c8ec-b653-4c50-8daa-fafa961caf89 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:25.204605 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf9c8ec-b653-4c50-8daa-fafa961caf89 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:25.204605 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 121/250] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:22:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 15:22:25.209532 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74e898f-e13c-446c-bcaa-8ca7bbabbcab None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:25.209831 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74e898f-e13c-446c-bcaa-8ca7bbabbcab None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:25.210118 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74e898f-e13c-446c-bcaa-8ca7bbabbcab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:25.228981 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-c74e898f-e13c-446c-bcaa-8ca7bbabbcab None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:22:25.230264 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c74e898f-e13c-446c-bcaa-8ca7bbabbcab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:22:25.313789 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 122/251] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:22:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:22:25.319901 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ad60c4-7944-4641-a6c7-12895664f934 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:25.352807 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74e898f-e13c-446c-bcaa-8ca7bbabbcab None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:25.361884 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69851119-d24e-4aad-9ee0-0a0588943d0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:25.362922 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69851119-d24e-4aad-9ee0-0a0588943d0c None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:25.363383 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69851119-d24e-4aad-9ee0-0a0588943d0c None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:25.363860 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69851119-d24e-4aad-9ee0-0a0588943d0c None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:25.364800 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69851119-d24e-4aad-9ee0-0a0588943d0c None admin] RBAC: Adding query filter params (name=member) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:22:25.365282 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69851119-d24e-4aad-9ee0-0a0588943d0c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:25.367139 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69851119-d24e-4aad-9ee0-0a0588943d0c None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:25.387636 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 130/252] 10.222.0.99 () {56 vars in 1101 bytes} [Thu Jul 14 15:22:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:25.388924 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69851119-d24e-4aad-9ee0-0a0588943d0c None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:25.406784 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:25.497809 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:25.522454 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74e898f-e13c-446c-bcaa-8ca7bbabbcab None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:25.566255 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99382537-0dd1-41d5-84a6-57bb37a00938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:25.567066 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99382537-0dd1-41d5-84a6-57bb37a00938 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:25.567573 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99382537-0dd1-41d5-84a6-57bb37a00938 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:25.567901 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99382537-0dd1-41d5-84a6-57bb37a00938 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:25.571330 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69851119-d24e-4aad-9ee0-0a0588943d0c None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:25.592152 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99382537-0dd1-41d5-84a6-57bb37a00938 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:22:25.600592 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-99382537-0dd1-41d5-84a6-57bb37a00938 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:22:25.600592 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:22:25.600592 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:22:25.600592 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:22:25.600592 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:22:25.600592 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:22:25.600592 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:22:25.600592 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:22:25.600592 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:22:25.600592 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:22:25.600592 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:22:25.600592 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:22:25.600592 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:22:25.600592 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:22:25.600592 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:22:25.600592 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:22:25.601484 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:22:25.601484 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:22:25.601484 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:22:25.601484 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:22:25.601484 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:22:25.601484 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:22:25.601484 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:22:25.601484 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:22:25.601484 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:22:25.601484 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:22:25.601484 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:22:25.601484 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:22:25.601484 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:22:25.601484 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:22:25.601484 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:22:25.601484 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:22:25.602276 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:22:25.602276 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:22:25.602276 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:22:25.602276 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:22:25.602276 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:22:25.602276 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:22:25.602276 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:22:25.602276 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:22:25.602276 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:22:25.602276 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:22:25.602276 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:22:25.602276 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:22:25.606865 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 123/253] 10.222.0.99 () {56 vars in 1096 bytes} [Thu Jul 14 15:22:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 15:22:25.608275 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99382537-0dd1-41d5-84a6-57bb37a00938 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:25.657298 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0461c8ba-326d-4a5b-882d-f5a6fd6472a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:25.658209 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0461c8ba-326d-4a5b-882d-f5a6fd6472a5 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:25.658499 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0461c8ba-326d-4a5b-882d-f5a6fd6472a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:25.658781 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0461c8ba-326d-4a5b-882d-f5a6fd6472a5 None admin] PATH_INFO: `/v3/users/3f2dd8e31dd1411281812a0144ac8816` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:25.660035 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0461c8ba-326d-4a5b-882d-f5a6fd6472a5 None admin] RBAC: Authorizing `identity:get_user(user_id=3f2dd8e31dd1411281812a0144ac8816)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:25.661509 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0461c8ba-326d-4a5b-882d-f5a6fd6472a5 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:25.668047 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 131/254] 10.222.0.99 () {56 vars in 1177 bytes} [Thu Jul 14 15:22:25 2022] GET /identity/v3/users/3f2dd8e31dd1411281812a0144ac8816 => generated 297 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:22:25.682367 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0461c8ba-326d-4a5b-882d-f5a6fd6472a5 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:25.709493 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:25.729481 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99382537-0dd1-41d5-84a6-57bb37a00938 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:25.775907 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6eddc9b-9aba-41c8-ae82-440915579646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:25.776600 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eddc9b-9aba-41c8-ae82-440915579646 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:25.776894 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eddc9b-9aba-41c8-ae82-440915579646 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:25.777177 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eddc9b-9aba-41c8-ae82-440915579646 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:25.777968 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6eddc9b-9aba-41c8-ae82-440915579646 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:22:25.778329 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6eddc9b-9aba-41c8-ae82-440915579646 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:25.779775 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6eddc9b-9aba-41c8-ae82-440915579646 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:25.795440 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 124/255] 10.222.0.99 () {56 vars in 1099 bytes} [Thu Jul 14 15:22:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:25.800148 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6eddc9b-9aba-41c8-ae82-440915579646 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:25.873579 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0461c8ba-326d-4a5b-882d-f5a6fd6472a5 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:25.937877 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b8de5c-c010-491f-8971-ce0c2fe6ad33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:25.938897 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b8de5c-c010-491f-8971-ce0c2fe6ad33 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:25.939259 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b8de5c-c010-491f-8971-ce0c2fe6ad33 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:25.939627 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b8de5c-c010-491f-8971-ce0c2fe6ad33 None admin] PATH_INFO: `/v3/projects/925bf9baf40d4a888a9d633d1deb5134` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:25.940952 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b8de5c-c010-491f-8971-ce0c2fe6ad33 None admin] RBAC: Authorizing `identity:get_project(project_id=925bf9baf40d4a888a9d633d1deb5134)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:25.942503 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b8de5c-c010-491f-8971-ce0c2fe6ad33 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:25.944505 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 132/256] 10.222.0.99 () {56 vars in 1186 bytes} [Thu Jul 14 15:22:25 2022] GET /identity/v3/projects/925bf9baf40d4a888a9d633d1deb5134 => generated 302 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:25.945633 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dfa489-36f5-4248-91db-21db7c1b191f None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:25.950717 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dfa489-36f5-4248-91db-21db7c1b191f None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:25.950717 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dfa489-36f5-4248-91db-21db7c1b191f None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:25.951048 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 133/257] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:22:25 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:22:25.951813 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dfa489-36f5-4248-91db-21db7c1b191f None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:25.972695 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:25.976480 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6eddc9b-9aba-41c8-ae82-440915579646 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:26.040885 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282b6a0a-ac04-4db9-bae3-ac3e95642224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:26.046836 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282b6a0a-ac04-4db9-bae3-ac3e95642224 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:26.047360 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282b6a0a-ac04-4db9-bae3-ac3e95642224 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:26.048386 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282b6a0a-ac04-4db9-bae3-ac3e95642224 None admin] PATH_INFO: `/v3/users/6ddb9f84c97f418a88ac32e44ba83caa` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:26.050189 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282b6a0a-ac04-4db9-bae3-ac3e95642224 None admin] RBAC: Authorizing `identity:get_user(user_id=6ddb9f84c97f418a88ac32e44ba83caa)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:26.051855 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282b6a0a-ac04-4db9-bae3-ac3e95642224 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:26.053874 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 125/258] 10.222.0.99 () {56 vars in 1177 bytes} [Thu Jul 14 15:22:25 2022] GET /identity/v3/users/6ddb9f84c97f418a88ac32e44ba83caa => generated 283 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:22:26.057299 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633c8363-d8ac-4940-9210-f168febc709f None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:26.059156 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633c8363-d8ac-4940-9210-f168febc709f None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:26.060327 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633c8363-d8ac-4940-9210-f168febc709f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:26.095236 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-633c8363-d8ac-4940-9210-f168febc709f None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:22:26.097308 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-633c8363-d8ac-4940-9210-f168febc709f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:22:26.118709 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:26.166325 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dfa489-36f5-4248-91db-21db7c1b191f None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:26.227567 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 126/259] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:22:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:22:26.229219 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633c8363-d8ac-4940-9210-f168febc709f None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:26.235323 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d8d3e9-5ba3-4aa6-86b9-2e9763cf7ec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:26.236474 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d8d3e9-5ba3-4aa6-86b9-2e9763cf7ec3 None admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:26.236940 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d8d3e9-5ba3-4aa6-86b9-2e9763cf7ec3 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:26.237392 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d8d3e9-5ba3-4aa6-86b9-2e9763cf7ec3 None admin] PATH_INFO: `/v3/projects/925bf9baf40d4a888a9d633d1deb5134/users/3f2dd8e31dd1411281812a0144ac8816/roles/b8c98fddc0fd4436b577fd1d3361bc38` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:26.239747 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d8d3e9-5ba3-4aa6-86b9-2e9763cf7ec3 None admin] RBAC: Authorizing `identity:create_grant(project_id=925bf9baf40d4a888a9d633d1deb5134, user_id=3f2dd8e31dd1411281812a0144ac8816, role_id=b8c98fddc0fd4436b577fd1d3361bc38)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:26.245782 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d8d3e9-5ba3-4aa6-86b9-2e9763cf7ec3 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:26.269059 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 134/260] 10.222.0.99 () {58 vars in 1439 bytes} [Thu Jul 14 15:22:25 2022] PUT /identity/v3/projects/925bf9baf40d4a888a9d633d1deb5134/users/3f2dd8e31dd1411281812a0144ac8816/roles/b8c98fddc0fd4436b577fd1d3361bc38 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:22:26.283089 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d8d3e9-5ba3-4aa6-86b9-2e9763cf7ec3 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:26.376091 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633c8363-d8ac-4940-9210-f168febc709f None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:26.402251 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:26.442149 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267dcb6f-98f1-4093-b1dc-7cb25962ef6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:26.443175 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267dcb6f-98f1-4093-b1dc-7cb25962ef6d None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:26.443690 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267dcb6f-98f1-4093-b1dc-7cb25962ef6d None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:26.444746 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267dcb6f-98f1-4093-b1dc-7cb25962ef6d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:26.471303 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d8d3e9-5ba3-4aa6-86b9-2e9763cf7ec3 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:26.472000 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:26.539632 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77aa2e7-4000-470a-a071-b6b73a520250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:26.542447 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77aa2e7-4000-470a-a071-b6b73a520250 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:26.542947 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77aa2e7-4000-470a-a071-b6b73a520250 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:26.543424 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77aa2e7-4000-470a-a071-b6b73a520250 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:26.564133 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77aa2e7-4000-470a-a071-b6b73a520250 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:22:26.574100 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267dcb6f-98f1-4093-b1dc-7cb25962ef6d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:26.575836 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267dcb6f-98f1-4093-b1dc-7cb25962ef6d None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:26.580143 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-c77aa2e7-4000-470a-a071-b6b73a520250 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 15:22:26.580143 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:22:26.580143 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:22:26.580143 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:22:26.580143 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:22:26.580143 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:22:26.580143 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:22:26.580143 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:22:26.580143 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:22:26.580143 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:22:26.580143 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:22:26.580143 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:22:26.580143 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:22:26.580143 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:22:26.580143 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:22:26.580143 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:22:26.581050 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:22:26.581050 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:22:26.581050 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:22:26.581050 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:22:26.581050 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:22:26.581050 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:22:26.581050 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:22:26.581050 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:22:26.581050 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:22:26.581050 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:22:26.581050 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:22:26.581050 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:22:26.581050 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:22:26.581050 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:22:26.581050 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:22:26.581050 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:22:26.581837 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:22:26.581837 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:22:26.581837 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:22:26.581837 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:22:26.581837 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:22:26.581837 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:22:26.581837 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:22:26.581837 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:22:26.581837 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:22:26.581837 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:22:26.581837 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 15:22:26.581837 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:22:26.585313 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 135/261] 10.222.0.99 () {56 vars in 1114 bytes} [Thu Jul 14 15:22:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:22:26.592932 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77aa2e7-4000-470a-a071-b6b73a520250 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:26.771976 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 127/262] 10.222.0.99 () {58 vars in 1282 bytes} [Thu Jul 14 15:22:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 543 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 15:22:26.784781 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267dcb6f-98f1-4093-b1dc-7cb25962ef6d None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:26.820793 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77aa2e7-4000-470a-a071-b6b73a520250 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:26.886607 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:26.902732 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:26.905374 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dca6790-57d4-4cf8-97df-831ea7eb46a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:26.906063 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dca6790-57d4-4cf8-97df-831ea7eb46a6 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:26.906347 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dca6790-57d4-4cf8-97df-831ea7eb46a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:26.906625 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dca6790-57d4-4cf8-97df-831ea7eb46a6 None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:26.907382 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dca6790-57d4-4cf8-97df-831ea7eb46a6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:22:26.907720 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dca6790-57d4-4cf8-97df-831ea7eb46a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:26.909294 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dca6790-57d4-4cf8-97df-831ea7eb46a6 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:26.929796 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 136/263] 10.222.0.99 () {56 vars in 1111 bytes} [Thu Jul 14 15:22:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:26.936243 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dca6790-57d4-4cf8-97df-831ea7eb46a6 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:27.004246 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267dcb6f-98f1-4093-b1dc-7cb25962ef6d None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:27.080672 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ab7d8e-1684-4c83-aff9-b3412f71b202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:27.081476 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ab7d8e-1684-4c83-aff9-b3412f71b202 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:27.081768 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ab7d8e-1684-4c83-aff9-b3412f71b202 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:27.082049 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ab7d8e-1684-4c83-aff9-b3412f71b202 None admin] PATH_INFO: `/v3/projects/925bf9baf40d4a888a9d633d1deb5134` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:27.085908 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ab7d8e-1684-4c83-aff9-b3412f71b202 None admin] RBAC: Authorizing `identity:get_project(project_id=925bf9baf40d4a888a9d633d1deb5134)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:27.087704 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ab7d8e-1684-4c83-aff9-b3412f71b202 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:27.090553 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 128/264] 10.222.0.99 () {56 vars in 1186 bytes} [Thu Jul 14 15:22:26 2022] GET /identity/v3/projects/925bf9baf40d4a888a9d633d1deb5134 => generated 302 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:27.096209 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ab7d8e-1684-4c83-aff9-b3412f71b202 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:27.165943 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dca6790-57d4-4cf8-97df-831ea7eb46a6 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:27.241661 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66d439f-69ad-4d38-87db-ba2180da899e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:27.242650 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66d439f-69ad-4d38-87db-ba2180da899e None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:27.242941 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66d439f-69ad-4d38-87db-ba2180da899e None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:27.243224 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66d439f-69ad-4d38-87db-ba2180da899e None admin] PATH_INFO: `/v3/users/6ddb9f84c97f418a88ac32e44ba83caa` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:27.245226 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ab7d8e-1684-4c83-aff9-b3412f71b202 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:27.247359 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66d439f-69ad-4d38-87db-ba2180da899e None admin] RBAC: Authorizing `identity:get_user(user_id=6ddb9f84c97f418a88ac32e44ba83caa)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:27.248965 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66d439f-69ad-4d38-87db-ba2180da899e None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:27.251184 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 137/265] 10.222.0.99 () {56 vars in 1177 bytes} [Thu Jul 14 15:22:26 2022] GET /identity/v3/users/6ddb9f84c97f418a88ac32e44ba83caa => generated 283 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:22:27.260877 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66d439f-69ad-4d38-87db-ba2180da899e None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:27.269152 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:27.303163 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24b3e27-c7eb-42b4-a82c-74ad3b60c790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:27.303898 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24b3e27-c7eb-42b4-a82c-74ad3b60c790 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:27.304164 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24b3e27-c7eb-42b4-a82c-74ad3b60c790 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:27.304450 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24b3e27-c7eb-42b4-a82c-74ad3b60c790 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:27.305304 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24b3e27-c7eb-42b4-a82c-74ad3b60c790 None admin] RBAC: Adding query filter params (role.id=d0f6ba1314f14d9f9c816bdb5d0c3aac, scope.project.id=925bf9baf40d4a888a9d633d1deb5134, user.id=6ddb9f84c97f418a88ac32e44ba83caa) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:22:27.305625 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24b3e27-c7eb-42b4-a82c-74ad3b60c790 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:27.307663 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24b3e27-c7eb-42b4-a82c-74ad3b60c790 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:27.319742 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 129/266] 10.222.0.99 () {56 vars in 1374 bytes} [Thu Jul 14 15:22:27 2022] GET /identity/v3/role_assignments?user.id=6ddb9f84c97f418a88ac32e44ba83caa&scope.project.id=925bf9baf40d4a888a9d633d1deb5134&role.id=d0f6ba1314f14d9f9c816bdb5d0c3aac => generated 608 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:27.404000 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:27.487039 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66d439f-69ad-4d38-87db-ba2180da899e None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:27.528170 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428966a6-0f51-48c1-b28d-b1f4bfc1873a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:27.528956 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428966a6-0f51-48c1-b28d-b1f4bfc1873a None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:27.529253 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428966a6-0f51-48c1-b28d-b1f4bfc1873a None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:27.529588 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428966a6-0f51-48c1-b28d-b1f4bfc1873a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:27.606074 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428966a6-0f51-48c1-b28d-b1f4bfc1873a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:27.607569 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428966a6-0f51-48c1-b28d-b1f4bfc1873a None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:27.669629 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:27.717463 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 138/267] 10.222.0.99 () {58 vars in 1282 bytes} [Thu Jul 14 15:22:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 15:22:27.723897 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24b3e27-c7eb-42b4-a82c-74ad3b60c790 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:27.851463 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24b3e27-c7eb-42b4-a82c-74ad3b60c790 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:27.888602 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917a1889-f3b9-4a28-9a08-0dbf4f470c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:27.889485 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917a1889-f3b9-4a28-9a08-0dbf4f470c98 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:27.889776 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917a1889-f3b9-4a28-9a08-0dbf4f470c98 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:27.890058 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917a1889-f3b9-4a28-9a08-0dbf4f470c98 None admin] PATH_INFO: `/v3/projects/925bf9baf40d4a888a9d633d1deb5134` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:27.891278 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917a1889-f3b9-4a28-9a08-0dbf4f470c98 None admin] RBAC: Authorizing `identity:get_project(project_id=925bf9baf40d4a888a9d633d1deb5134)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:27.892802 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917a1889-f3b9-4a28-9a08-0dbf4f470c98 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:27.896712 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 130/268] 10.222.0.99 () {56 vars in 1186 bytes} [Thu Jul 14 15:22:27 2022] GET /identity/v3/projects/925bf9baf40d4a888a9d633d1deb5134 => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:27.901839 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428966a6-0f51-48c1-b28d-b1f4bfc1873a None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:28.028302 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428966a6-0f51-48c1-b28d-b1f4bfc1873a None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:28.070347 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ffc461-f220-4aad-aacc-89295a3a14a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:28.071027 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ffc461-f220-4aad-aacc-89295a3a14a3 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:28.071319 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ffc461-f220-4aad-aacc-89295a3a14a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:28.071620 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ffc461-f220-4aad-aacc-89295a3a14a3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:28.072489 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ffc461-f220-4aad-aacc-89295a3a14a3 None admin] RBAC: Adding query filter params (role.id=3627b4f45c8c4f41a7def6b15f1b8225, scope.project.id=925bf9baf40d4a888a9d633d1deb5134, user.id=6ddb9f84c97f418a88ac32e44ba83caa) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:22:28.072851 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ffc461-f220-4aad-aacc-89295a3a14a3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:28.074317 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ffc461-f220-4aad-aacc-89295a3a14a3 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:28.080975 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 139/269] 10.222.0.99 () {56 vars in 1374 bytes} [Thu Jul 14 15:22:27 2022] GET /identity/v3/role_assignments?user.id=6ddb9f84c97f418a88ac32e44ba83caa&scope.project.id=925bf9baf40d4a888a9d633d1deb5134&role.id=3627b4f45c8c4f41a7def6b15f1b8225 => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:28.163711 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465e0888-96d3-41bc-9b5d-99ae7e7f07ae None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:28.164346 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465e0888-96d3-41bc-9b5d-99ae7e7f07ae None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:28.164855 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465e0888-96d3-41bc-9b5d-99ae7e7f07ae None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:28.166925 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 131/270] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:22:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:22:28.173399 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b282fcf-2fbd-48ef-a847-d2a86034acf8 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:28.173693 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b282fcf-2fbd-48ef-a847-d2a86034acf8 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:28.174144 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b282fcf-2fbd-48ef-a847-d2a86034acf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:28.196493 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-2b282fcf-2fbd-48ef-a847-d2a86034acf8 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:22:28.198666 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-2b282fcf-2fbd-48ef-a847-d2a86034acf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:22:28.218897 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:28.278220 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8fdb9f-e48f-4351-b747-bfccea62504c None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:28.278553 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8fdb9f-e48f-4351-b747-bfccea62504c None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:28.278721 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8fdb9f-e48f-4351-b747-bfccea62504c None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:28.280292 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 132/271] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:22:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 15:22:28.298022 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9541775e-58a1-40a1-9720-2358cfe8ef74 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:28.298541 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9541775e-58a1-40a1-9720-2358cfe8ef74 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:28.298994 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9541775e-58a1-40a1-9720-2358cfe8ef74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:28.306383 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 140/272] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:22:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:22:28.309416 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed8abaf-183a-4409-a0b8-742c8a03783f None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:28.309715 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed8abaf-183a-4409-a0b8-742c8a03783f None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:28.310001 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed8abaf-183a-4409-a0b8-742c8a03783f None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:28.311413 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 141/273] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:22:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:22:28.328431 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9ba1ad-e81d-4bf6-8fb8-a56d980ed4ea None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:28.328729 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9ba1ad-e81d-4bf6-8fb8-a56d980ed4ea None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:28.329019 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9ba1ad-e81d-4bf6-8fb8-a56d980ed4ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:28.334264 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-9541775e-58a1-40a1-9720-2358cfe8ef74 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:22:28.335698 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-9541775e-58a1-40a1-9720-2358cfe8ef74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:22:28.348855 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-1f9ba1ad-e81d-4bf6-8fb8-a56d980ed4ea None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:22:28.350201 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-1f9ba1ad-e81d-4bf6-8fb8-a56d980ed4ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:22:28.383158 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:28.427620 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 133/274] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:22:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 15:22:28.427620 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9541775e-58a1-40a1-9720-2358cfe8ef74 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:28.445314 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 142/275] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:22:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 15:22:28.455623 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9ba1ad-e81d-4bf6-8fb8-a56d980ed4ea None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:28.550133 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9541775e-58a1-40a1-9720-2358cfe8ef74 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:28.560243 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:28.581781 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9ba1ad-e81d-4bf6-8fb8-a56d980ed4ea None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:28.587035 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42918e6-89f7-4592-91d3-220d6fb10a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:28.587869 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42918e6-89f7-4592-91d3-220d6fb10a88 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:28.588162 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42918e6-89f7-4592-91d3-220d6fb10a88 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:28.588490 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42918e6-89f7-4592-91d3-220d6fb10a88 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:28.600265 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42918e6-89f7-4592-91d3-220d6fb10a88 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:22:28.608233 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-f42918e6-89f7-4592-91d3-220d6fb10a88 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 15:22:28.608233 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:22:28.608233 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:22:28.608233 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:22:28.608233 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:22:28.608233 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:22:28.608233 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:22:28.608233 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:22:28.608233 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:22:28.608233 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:22:28.608233 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:22:28.608233 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:22:28.608233 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:22:28.608233 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:22:28.608233 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:22:28.608233 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:22:28.609150 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:22:28.609150 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:22:28.609150 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:22:28.609150 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:22:28.609150 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:22:28.609150 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:22:28.609150 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:22:28.609150 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:22:28.609150 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:22:28.609150 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:22:28.609150 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:22:28.609150 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:22:28.609150 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:22:28.609150 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:22:28.609150 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:22:28.609150 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:22:28.609921 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:22:28.609921 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:22:28.609921 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:22:28.609921 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:22:28.609921 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:22:28.609921 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:22:28.609921 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:22:28.609921 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:22:28.609921 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:22:28.609921 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:22:28.609921 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 15:22:28.609921 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:22:28.609921 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 134/276] 10.222.0.99 () {56 vars in 1099 bytes} [Thu Jul 14 15:22:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:22:28.610570 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42918e6-89f7-4592-91d3-220d6fb10a88 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:28.631696 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4355ca-0591-4ed2-bc43-04fd543696e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:28.632556 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4355ca-0591-4ed2-bc43-04fd543696e7 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:28.633123 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4355ca-0591-4ed2-bc43-04fd543696e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:28.633123 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4355ca-0591-4ed2-bc43-04fd543696e7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:28.634302 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4355ca-0591-4ed2-bc43-04fd543696e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:28.636367 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4355ca-0591-4ed2-bc43-04fd543696e7 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:28.638186 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 143/277] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:22:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:22:28.643443 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4355ca-0591-4ed2-bc43-04fd543696e7 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:28.660620 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:28.734836 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42918e6-89f7-4592-91d3-220d6fb10a88 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:28.755334 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4355ca-0591-4ed2-bc43-04fd543696e7 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:28.770197 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bc6cfb-c827-4d70-82ef-bf2b80a332e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:28.770999 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bc6cfb-c827-4d70-82ef-bf2b80a332e6 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:28.771287 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bc6cfb-c827-4d70-82ef-bf2b80a332e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:28.771567 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bc6cfb-c827-4d70-82ef-bf2b80a332e6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:28.782444 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bc6cfb-c827-4d70-82ef-bf2b80a332e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:22:28.790372 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-55bc6cfb-c827-4d70-82ef-bf2b80a332e6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 15:22:28.790372 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:22:28.790372 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:22:28.790372 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:22:28.790372 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:22:28.790372 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:22:28.790372 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:22:28.790372 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:22:28.790372 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:22:28.790372 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:22:28.790372 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:22:28.790372 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:22:28.790372 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:22:28.790372 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:22:28.790372 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:22:28.790372 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:22:28.791347 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:22:28.791347 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:22:28.791347 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:22:28.791347 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:22:28.791347 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:22:28.791347 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:22:28.791347 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:22:28.791347 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:22:28.791347 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:22:28.791347 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:22:28.791347 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:22:28.791347 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:22:28.791347 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:22:28.791347 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:22:28.791347 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:22:28.791347 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:22:28.793159 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:22:28.793159 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:22:28.793159 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:22:28.793159 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:22:28.793159 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:22:28.793159 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:22:28.793159 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:22:28.793159 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:22:28.793159 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:22:28.793159 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:22:28.793159 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 15:22:28.793159 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:22:28.793159 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 135/278] 10.222.0.99 () {56 vars in 1099 bytes} [Thu Jul 14 15:22:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:22:28.793159 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bc6cfb-c827-4d70-82ef-bf2b80a332e6 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:28.797127 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c37f81-352f-470c-bb6a-6f48b5e8058c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:28.797819 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c37f81-352f-470c-bb6a-6f48b5e8058c None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:28.798114 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c37f81-352f-470c-bb6a-6f48b5e8058c None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:28.798399 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c37f81-352f-470c-bb6a-6f48b5e8058c None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:28.799174 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c37f81-352f-470c-bb6a-6f48b5e8058c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:22:28.799486 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c37f81-352f-470c-bb6a-6f48b5e8058c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:28.800972 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c37f81-352f-470c-bb6a-6f48b5e8058c None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:28.815442 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 144/279] 10.222.0.99 () {56 vars in 1101 bytes} [Thu Jul 14 15:22:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:28.815442 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c37f81-352f-470c-bb6a-6f48b5e8058c None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:28.831251 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:28.902054 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:28.911001 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bc6cfb-c827-4d70-82ef-bf2b80a332e6 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:28.930181 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c37f81-352f-470c-bb6a-6f48b5e8058c None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:28.947096 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a972ee-7c8a-49c2-a18b-ed6faaf113a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:28.947860 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a972ee-7c8a-49c2-a18b-ed6faaf113a6 None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:28.948634 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a972ee-7c8a-49c2-a18b-ed6faaf113a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:28.949159 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a972ee-7c8a-49c2-a18b-ed6faaf113a6 None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:28.949661 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a972ee-7c8a-49c2-a18b-ed6faaf113a6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:28.951110 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a972ee-7c8a-49c2-a18b-ed6faaf113a6 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:28.971762 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d52a97-5adc-407a-ae51-f6b5b861969a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:28.972004 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 136/280] 10.222.0.99 () {60 vars in 1132 bytes} [Thu Jul 14 15:22:28 2022] POST /identity/v3/users => generated 293 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:22:28.972454 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d52a97-5adc-407a-ae51-f6b5b861969a None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:28.972716 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d52a97-5adc-407a-ae51-f6b5b861969a None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:28.972826 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a972ee-7c8a-49c2-a18b-ed6faaf113a6 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:28.973001 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d52a97-5adc-407a-ae51-f6b5b861969a None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:28.973729 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d52a97-5adc-407a-ae51-f6b5b861969a None admin] RBAC: Adding query filter params (name=member) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:22:28.974036 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d52a97-5adc-407a-ae51-f6b5b861969a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:28.975433 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d52a97-5adc-407a-ae51-f6b5b861969a None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:28.985510 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 145/281] 10.222.0.99 () {56 vars in 1101 bytes} [Thu Jul 14 15:22:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:28.986882 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab5ebdf-9fd6-4ace-a3e9-d989b910f883 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:28.987176 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab5ebdf-9fd6-4ace-a3e9-d989b910f883 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:28.987456 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab5ebdf-9fd6-4ace-a3e9-d989b910f883 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:28.988741 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 146/282] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:22:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:22:28.997330 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab5ebdf-9fd6-4ace-a3e9-d989b910f883 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:29.088103 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:29.115954 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a972ee-7c8a-49c2-a18b-ed6faaf113a6 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:29.127400 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab5ebdf-9fd6-4ace-a3e9-d989b910f883 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:29.162946 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de6d1de-bf24-46c4-9148-966ee265bae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:29.164013 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de6d1de-bf24-46c4-9148-966ee265bae3 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:29.164305 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de6d1de-bf24-46c4-9148-966ee265bae3 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:29.164586 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de6d1de-bf24-46c4-9148-966ee265bae3 None admin] PATH_INFO: `/v3/users/c54065f8d99f443a8e0550a404b05514` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:29.165875 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de6d1de-bf24-46c4-9148-966ee265bae3 None admin] RBAC: Authorizing `identity:get_user(user_id=c54065f8d99f443a8e0550a404b05514)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:29.167618 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de6d1de-bf24-46c4-9148-966ee265bae3 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:29.169290 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 137/283] 10.222.0.99 () {56 vars in 1177 bytes} [Thu Jul 14 15:22:28 2022] GET /identity/v3/users/c54065f8d99f443a8e0550a404b05514 => generated 297 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:29.170459 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf075a2-6a81-420a-9cf8-9b67b4d4ca24 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:29.170745 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf075a2-6a81-420a-9cf8-9b67b4d4ca24 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:29.171024 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf075a2-6a81-420a-9cf8-9b67b4d4ca24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:29.178277 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbd4283-ea21-4461-baa2-dea4de1e534b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:29.179194 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbd4283-ea21-4461-baa2-dea4de1e534b None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:29.179494 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbd4283-ea21-4461-baa2-dea4de1e534b None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:29.179804 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbd4283-ea21-4461-baa2-dea4de1e534b None admin] PATH_INFO: `/v3/users/865db4991e9941c9a7f33c23c1190fa3` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:29.189662 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-faf075a2-6a81-420a-9cf8-9b67b4d4ca24 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:22:29.191520 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-faf075a2-6a81-420a-9cf8-9b67b4d4ca24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:22:29.200570 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbd4283-ea21-4461-baa2-dea4de1e534b None admin] RBAC: Authorizing `identity:get_user(user_id=865db4991e9941c9a7f33c23c1190fa3)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:29.202197 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbd4283-ea21-4461-baa2-dea4de1e534b None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:29.210530 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 147/284] 10.222.0.99 () {56 vars in 1177 bytes} [Thu Jul 14 15:22:28 2022] GET /identity/v3/users/865db4991e9941c9a7f33c23c1190fa3 => generated 293 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:29.210530 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbd4283-ea21-4461-baa2-dea4de1e534b None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:29.213231 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:29.308003 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 138/285] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:22:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:22:29.308876 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf075a2-6a81-420a-9cf8-9b67b4d4ca24 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:29.352071 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbd4283-ea21-4461-baa2-dea4de1e534b None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:29.383759 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:29.395671 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbdff48-b290-453e-bde2-625fa14533e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:29.395671 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbdff48-b290-453e-bde2-625fa14533e6 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:29.395671 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbdff48-b290-453e-bde2-625fa14533e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:29.396005 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbdff48-b290-453e-bde2-625fa14533e6 None admin] PATH_INFO: `/v3/projects/925bf9baf40d4a888a9d633d1deb5134` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:29.399618 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbdff48-b290-453e-bde2-625fa14533e6 None admin] RBAC: Authorizing `identity:get_project(project_id=925bf9baf40d4a888a9d633d1deb5134)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:29.399618 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbdff48-b290-453e-bde2-625fa14533e6 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:29.403510 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 148/286] 10.222.0.99 () {56 vars in 1186 bytes} [Thu Jul 14 15:22:29 2022] GET /identity/v3/projects/925bf9baf40d4a888a9d633d1deb5134 => generated 302 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:29.407164 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbdff48-b290-453e-bde2-625fa14533e6 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:29.511626 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf075a2-6a81-420a-9cf8-9b67b4d4ca24 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:29.591625 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f120fea-6f1e-4fbc-afd7-efe693a2ebe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:29.591625 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f120fea-6f1e-4fbc-afd7-efe693a2ebe4 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:29.591625 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f120fea-6f1e-4fbc-afd7-efe693a2ebe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:29.591625 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f120fea-6f1e-4fbc-afd7-efe693a2ebe4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:29.592029 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f120fea-6f1e-4fbc-afd7-efe693a2ebe4 None admin] RBAC: Adding query filter params (role.id=b8c98fddc0fd4436b577fd1d3361bc38, user.id=865db4991e9941c9a7f33c23c1190fa3, scope.system=all) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:22:29.592029 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f120fea-6f1e-4fbc-afd7-efe693a2ebe4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:29.592623 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f120fea-6f1e-4fbc-afd7-efe693a2ebe4 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:29.595333 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbdff48-b290-453e-bde2-625fa14533e6 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:29.601945 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 139/287] 10.222.0.99 () {56 vars in 1308 bytes} [Thu Jul 14 15:22:29 2022] GET /identity/v3/role_assignments?user.id=865db4991e9941c9a7f33c23c1190fa3&scope.system=all&role.id=b8c98fddc0fd4436b577fd1d3361bc38 => generated 226 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:29.602898 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f120fea-6f1e-4fbc-afd7-efe693a2ebe4 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:29.655304 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:29.704701 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0835534-751d-400d-bba2-a41e4cd4b695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:29.705558 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0835534-751d-400d-bba2-a41e4cd4b695 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:29.705851 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0835534-751d-400d-bba2-a41e4cd4b695 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:29.706133 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0835534-751d-400d-bba2-a41e4cd4b695 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:29.751631 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0835534-751d-400d-bba2-a41e4cd4b695 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:22:29.756387 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-f0835534-751d-400d-bba2-a41e4cd4b695 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 15:22:29.756387 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:22:29.756387 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:22:29.756387 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:22:29.756387 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:22:29.756387 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:22:29.756387 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:22:29.756387 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:22:29.756387 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:22:29.756387 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:22:29.756387 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:22:29.756387 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:22:29.756387 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:22:29.756387 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:22:29.756387 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:22:29.756387 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:22:29.757321 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:22:29.757321 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:22:29.757321 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:22:29.757321 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:22:29.757321 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:22:29.757321 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:22:29.757321 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:22:29.757321 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:22:29.757321 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:22:29.757321 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:22:29.757321 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:22:29.757321 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:22:29.757321 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:22:29.757321 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:22:29.757321 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:22:29.757321 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:22:29.758101 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:22:29.758101 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:22:29.758101 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:22:29.758101 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:22:29.758101 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:22:29.758101 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:22:29.758101 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:22:29.758101 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:22:29.758101 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:22:29.758101 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:22:29.758101 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 15:22:29.758101 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:22:29.761474 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 149/288] 10.222.0.99 () {56 vars in 1099 bytes} [Thu Jul 14 15:22:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 357 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:22:29.770093 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0835534-751d-400d-bba2-a41e4cd4b695 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:29.892455 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:29.907067 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f120fea-6f1e-4fbc-afd7-efe693a2ebe4 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:29.970822 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0835534-751d-400d-bba2-a41e4cd4b695 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:29.976230 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d79aba9-a21e-4951-a6ec-a0e5bb3d5fb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:29.977129 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d79aba9-a21e-4951-a6ec-a0e5bb3d5fb1 None admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:29.977412 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d79aba9-a21e-4951-a6ec-a0e5bb3d5fb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:29.977690 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d79aba9-a21e-4951-a6ec-a0e5bb3d5fb1 None admin] PATH_INFO: `/v3/projects/925bf9baf40d4a888a9d633d1deb5134/users/c54065f8d99f443a8e0550a404b05514/roles/cc5c3091b148423684e5e966150bc6c6` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:29.981331 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d79aba9-a21e-4951-a6ec-a0e5bb3d5fb1 None admin] RBAC: Authorizing `identity:create_grant(project_id=925bf9baf40d4a888a9d633d1deb5134, user_id=c54065f8d99f443a8e0550a404b05514, role_id=cc5c3091b148423684e5e966150bc6c6)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:29.982798 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d79aba9-a21e-4951-a6ec-a0e5bb3d5fb1 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:30.012104 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 140/289] 10.222.0.99 () {58 vars in 1439 bytes} [Thu Jul 14 15:22:29 2022] PUT /identity/v3/projects/925bf9baf40d4a888a9d633d1deb5134/users/c54065f8d99f443a8e0550a404b05514/roles/cc5c3091b148423684e5e966150bc6c6 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:22:30.046050 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5057c2e7-8274-4387-88a2-692663d1049f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:30.047116 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5057c2e7-8274-4387-88a2-692663d1049f None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:30.047567 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5057c2e7-8274-4387-88a2-692663d1049f None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:30.048028 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5057c2e7-8274-4387-88a2-692663d1049f None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:30.049322 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5057c2e7-8274-4387-88a2-692663d1049f None admin] RBAC: Adding query filter params (name=member) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:22:30.049832 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5057c2e7-8274-4387-88a2-692663d1049f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:30.052401 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5057c2e7-8274-4387-88a2-692663d1049f None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:30.068502 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 150/290] 10.222.0.99 () {56 vars in 1101 bytes} [Thu Jul 14 15:22:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:30.074443 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d79aba9-a21e-4951-a6ec-a0e5bb3d5fb1 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:30.184903 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:30.225476 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d79aba9-a21e-4951-a6ec-a0e5bb3d5fb1 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:30.279040 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bd5fc1-a553-44c9-8a41-fce741d73224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:30.279966 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bd5fc1-a553-44c9-8a41-fce741d73224 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:30.280255 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bd5fc1-a553-44c9-8a41-fce741d73224 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:30.280530 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bd5fc1-a553-44c9-8a41-fce741d73224 None admin] PATH_INFO: `/v3/users/3f2dd8e31dd1411281812a0144ac8816` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:30.281796 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bd5fc1-a553-44c9-8a41-fce741d73224 None admin] RBAC: Authorizing `identity:get_user(user_id=3f2dd8e31dd1411281812a0144ac8816)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:30.283261 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bd5fc1-a553-44c9-8a41-fce741d73224 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:30.285005 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 141/291] 10.222.0.99 () {56 vars in 1177 bytes} [Thu Jul 14 15:22:30 2022] GET /identity/v3/users/3f2dd8e31dd1411281812a0144ac8816 => generated 297 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:30.293275 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5057c2e7-8274-4387-88a2-692663d1049f None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:30.414288 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5057c2e7-8274-4387-88a2-692663d1049f None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:30.440267 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:30.464565 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d93fb83-ca72-4017-b1a4-858d88347334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:30.465256 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d93fb83-ca72-4017-b1a4-858d88347334 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:30.465551 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d93fb83-ca72-4017-b1a4-858d88347334 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:30.465824 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d93fb83-ca72-4017-b1a4-858d88347334 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:30.576965 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d93fb83-ca72-4017-b1a4-858d88347334 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:30.578510 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d93fb83-ca72-4017-b1a4-858d88347334 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:30.712117 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 151/292] 10.222.0.99 () {58 vars in 1282 bytes} [Thu Jul 14 15:22:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 15:22:30.718344 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bd5fc1-a553-44c9-8a41-fce741d73224 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:30.836421 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bd5fc1-a553-44c9-8a41-fce741d73224 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:30.879366 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94916390-7a9f-458f-a2a1-1000ac602ac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:30.880281 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94916390-7a9f-458f-a2a1-1000ac602ac6 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:30.880569 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94916390-7a9f-458f-a2a1-1000ac602ac6 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:30.880852 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94916390-7a9f-458f-a2a1-1000ac602ac6 None admin] PATH_INFO: `/v3/projects/925bf9baf40d4a888a9d633d1deb5134` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:30.882078 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94916390-7a9f-458f-a2a1-1000ac602ac6 None admin] RBAC: Authorizing `identity:get_project(project_id=925bf9baf40d4a888a9d633d1deb5134)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:30.883541 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94916390-7a9f-458f-a2a1-1000ac602ac6 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:30.885282 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 142/293] 10.222.0.99 () {56 vars in 1186 bytes} [Thu Jul 14 15:22:30 2022] GET /identity/v3/projects/925bf9baf40d4a888a9d633d1deb5134 => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:30.891530 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d93fb83-ca72-4017-b1a4-858d88347334 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:30.920407 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:31.046477 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d93fb83-ca72-4017-b1a4-858d88347334 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:31.101329 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84c2059-bd1b-4b76-8c7e-4b706eb81db3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:31.102016 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84c2059-bd1b-4b76-8c7e-4b706eb81db3 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:31.102304 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84c2059-bd1b-4b76-8c7e-4b706eb81db3 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:31.102581 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84c2059-bd1b-4b76-8c7e-4b706eb81db3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:31.103429 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84c2059-bd1b-4b76-8c7e-4b706eb81db3 None admin] RBAC: Adding query filter params (role.id=b8c98fddc0fd4436b577fd1d3361bc38, scope.project.id=925bf9baf40d4a888a9d633d1deb5134, user.id=3f2dd8e31dd1411281812a0144ac8816) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:22:31.103772 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84c2059-bd1b-4b76-8c7e-4b706eb81db3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:31.106654 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84c2059-bd1b-4b76-8c7e-4b706eb81db3 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:31.116739 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 152/294] 10.222.0.99 () {56 vars in 1374 bytes} [Thu Jul 14 15:22:30 2022] GET /identity/v3/role_assignments?user.id=3f2dd8e31dd1411281812a0144ac8816&scope.project.id=925bf9baf40d4a888a9d633d1deb5134&role.id=b8c98fddc0fd4436b577fd1d3361bc38 => generated 608 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:31.714500 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e3147e-9415-4995-8fdb-8d49773def02 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:31.715023 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e3147e-9415-4995-8fdb-8d49773def02 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:31.715098 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e3147e-9415-4995-8fdb-8d49773def02 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:31.716774 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 143/295] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:22:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:22:31.721807 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6084aaf3-55ab-47ac-b9d5-a3fff750b613 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:31.722368 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6084aaf3-55ab-47ac-b9d5-a3fff750b613 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:31.722869 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6084aaf3-55ab-47ac-b9d5-a3fff750b613 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:31.739689 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0d702a-ce60-46d4-8dae-1ad07691321a None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:31.740313 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0d702a-ce60-46d4-8dae-1ad07691321a None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:31.740789 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0d702a-ce60-46d4-8dae-1ad07691321a None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:31.742428 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 144/296] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:22:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 15:22:31.747629 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-6084aaf3-55ab-47ac-b9d5-a3fff750b613 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:22:31.747629 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-6084aaf3-55ab-47ac-b9d5-a3fff750b613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:22:31.750412 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c801618b-00b0-4ef7-999f-69fba3ca1f8a None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:31.750955 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c801618b-00b0-4ef7-999f-69fba3ca1f8a None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:31.752373 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c801618b-00b0-4ef7-999f-69fba3ca1f8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:31.773344 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-c801618b-00b0-4ef7-999f-69fba3ca1f8a None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:22:31.775177 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c801618b-00b0-4ef7-999f-69fba3ca1f8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:22:31.778434 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:31.839954 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 153/297] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:22:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:22:31.868484 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 145/298] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:22:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:22:31.868950 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6084aaf3-55ab-47ac-b9d5-a3fff750b613 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:31.886731 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:31.894444 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c801618b-00b0-4ef7-999f-69fba3ca1f8a None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:32.003571 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6084aaf3-55ab-47ac-b9d5-a3fff750b613 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:32.014585 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c801618b-00b0-4ef7-999f-69fba3ca1f8a None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:32.053011 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77544603-d03a-4e79-8695-ac4ed2aa6120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:32.054683 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77544603-d03a-4e79-8695-ac4ed2aa6120 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:32.055171 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77544603-d03a-4e79-8695-ac4ed2aa6120 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:32.055698 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77544603-d03a-4e79-8695-ac4ed2aa6120 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:32.056863 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267c243a-a53c-4e86-bbb6-b0d810d15a42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:32.057351 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77544603-d03a-4e79-8695-ac4ed2aa6120 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:32.058362 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267c243a-a53c-4e86-bbb6-b0d810d15a42 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:32.058650 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267c243a-a53c-4e86-bbb6-b0d810d15a42 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:32.058922 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267c243a-a53c-4e86-bbb6-b0d810d15a42 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:32.060481 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77544603-d03a-4e79-8695-ac4ed2aa6120 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:32.063147 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 154/299] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:22:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:22:32.068958 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77544603-d03a-4e79-8695-ac4ed2aa6120 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:32.070334 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:32.075342 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267c243a-a53c-4e86-bbb6-b0d810d15a42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:22:32.085114 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-267c243a-a53c-4e86-bbb6-b0d810d15a42 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 15:22:32.085114 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:22:32.085114 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:22:32.085114 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:22:32.085114 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:22:32.085114 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:22:32.085114 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:22:32.085114 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:22:32.085114 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:22:32.085114 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:22:32.085114 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:22:32.085114 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:22:32.085114 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:22:32.085114 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:22:32.085114 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:22:32.085114 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:22:32.086073 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:22:32.086073 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:22:32.086073 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:22:32.086073 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:22:32.086073 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:22:32.086073 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:22:32.086073 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:22:32.086073 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:22:32.086073 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:22:32.086073 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:22:32.086073 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:22:32.086073 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:22:32.086073 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:22:32.086073 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:22:32.086073 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:22:32.086073 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:22:32.086846 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:22:32.086846 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:22:32.086846 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:22:32.086846 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:22:32.086846 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:22:32.086846 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:22:32.086846 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:22:32.086846 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:22:32.086846 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:22:32.086846 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:22:32.086846 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 15:22:32.086846 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:22:32.086846 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 146/300] 10.222.0.99 () {56 vars in 1099 bytes} [Thu Jul 14 15:22:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 15:22:32.092216 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267c243a-a53c-4e86-bbb6-b0d810d15a42 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:32.169441 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:32.190776 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77544603-d03a-4e79-8695-ac4ed2aa6120 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:32.205616 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267c243a-a53c-4e86-bbb6-b0d810d15a42 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:32.227221 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ecfb03-f350-4191-8b96-26940c028a92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:32.228220 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ecfb03-f350-4191-8b96-26940c028a92 None admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:32.228503 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ecfb03-f350-4191-8b96-26940c028a92 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:32.228805 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ecfb03-f350-4191-8b96-26940c028a92 None admin] PATH_INFO: `/v3/groups` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:32.229988 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ecfb03-f350-4191-8b96-26940c028a92 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:32.231432 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ecfb03-f350-4191-8b96-26940c028a92 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:32.242171 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17e44c7-2fe9-4ee0-8a11-a38a67dfdd00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:32.242854 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17e44c7-2fe9-4ee0-8a11-a38a67dfdd00 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:32.243141 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17e44c7-2fe9-4ee0-8a11-a38a67dfdd00 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:32.243425 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17e44c7-2fe9-4ee0-8a11-a38a67dfdd00 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:32.244217 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17e44c7-2fe9-4ee0-8a11-a38a67dfdd00 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:22:32.244579 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17e44c7-2fe9-4ee0-8a11-a38a67dfdd00 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:32.246587 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17e44c7-2fe9-4ee0-8a11-a38a67dfdd00 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:32.247906 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 155/301] 10.222.0.99 () {60 vars in 1134 bytes} [Thu Jul 14 15:22:32 2022] POST /identity/v3/groups => generated 229 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 15:22:32.249219 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcb3b68-5c54-4fb2-ab40-6c9ab4c37579 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:32.249501 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcb3b68-5c54-4fb2-ab40-6c9ab4c37579 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:32.249779 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcb3b68-5c54-4fb2-ab40-6c9ab4c37579 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:32.251160 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 156/302] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:22:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:22:32.257080 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccbcf2b-2746-47a0-acc6-7215991e50c3 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:32.257393 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccbcf2b-2746-47a0-acc6-7215991e50c3 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:32.257643 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccbcf2b-2746-47a0-acc6-7215991e50c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:32.265005 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 147/303] 10.222.0.99 () {56 vars in 1101 bytes} [Thu Jul 14 15:22:32 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:32.272810 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17e44c7-2fe9-4ee0-8a11-a38a67dfdd00 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:32.282624 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-4ccbcf2b-2746-47a0-acc6-7215991e50c3 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:22:32.284346 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-4ccbcf2b-2746-47a0-acc6-7215991e50c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:22:32.348786 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:32.394026 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 157/304] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:22:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 15:22:32.400367 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17e44c7-2fe9-4ee0-8a11-a38a67dfdd00 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:32.417499 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccbcf2b-2746-47a0-acc6-7215991e50c3 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:32.449595 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c52e8d6-7ed3-45bc-b2d1-86ce138703fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:32.450604 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c52e8d6-7ed3-45bc-b2d1-86ce138703fa None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:32.450890 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c52e8d6-7ed3-45bc-b2d1-86ce138703fa None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:32.451169 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c52e8d6-7ed3-45bc-b2d1-86ce138703fa None admin] PATH_INFO: `/v3/users/77b3d29b19584ac29e46c2729c7726b0` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:32.467071 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c52e8d6-7ed3-45bc-b2d1-86ce138703fa None admin] RBAC: Authorizing `identity:get_user(user_id=77b3d29b19584ac29e46c2729c7726b0)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:32.468577 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c52e8d6-7ed3-45bc-b2d1-86ce138703fa None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:32.470530 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 148/305] 10.222.0.99 () {56 vars in 1177 bytes} [Thu Jul 14 15:22:32 2022] GET /identity/v3/users/77b3d29b19584ac29e46c2729c7726b0 => generated 293 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:32.476555 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c52e8d6-7ed3-45bc-b2d1-86ce138703fa None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:32.506294 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:32.532901 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccbcf2b-2746-47a0-acc6-7215991e50c3 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:32.570176 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93da67b9-2f0c-446d-b884-8a39e721ad4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:32.572047 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93da67b9-2f0c-446d-b884-8a39e721ad4a None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:32.572504 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93da67b9-2f0c-446d-b884-8a39e721ad4a None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:32.572953 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93da67b9-2f0c-446d-b884-8a39e721ad4a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:32.586400 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93da67b9-2f0c-446d-b884-8a39e721ad4a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:22:32.597181 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-93da67b9-2f0c-446d-b884-8a39e721ad4a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 15:22:32.597181 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:22:32.597181 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:22:32.597181 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:22:32.597181 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:22:32.597181 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:22:32.597181 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:22:32.597181 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:22:32.597181 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:22:32.597181 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:22:32.597181 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:22:32.597181 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:22:32.597181 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:22:32.597181 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:22:32.597181 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:22:32.597181 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:22:32.598270 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:22:32.598270 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:22:32.598270 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:22:32.598270 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:22:32.598270 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:22:32.598270 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:22:32.598270 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:22:32.598270 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:22:32.598270 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:22:32.598270 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:22:32.598270 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:22:32.598270 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:22:32.598270 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:22:32.598270 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:22:32.598270 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:22:32.598270 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:22:32.599226 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:22:32.599226 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:22:32.599226 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:22:32.599226 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:22:32.599226 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:22:32.599226 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:22:32.599226 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:22:32.599226 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:22:32.599226 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:22:32.599226 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:22:32.599226 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 15:22:32.599226 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:22:32.602304 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 158/306] 10.222.0.99 () {56 vars in 1099 bytes} [Thu Jul 14 15:22:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 15:22:32.609867 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93da67b9-2f0c-446d-b884-8a39e721ad4a None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:32.611950 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c52e8d6-7ed3-45bc-b2d1-86ce138703fa None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:32.631546 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:32.649614 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc80fa35-ff2f-4a1f-8d44-b82ab329622a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:32.650609 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc80fa35-ff2f-4a1f-8d44-b82ab329622a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:32.651038 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc80fa35-ff2f-4a1f-8d44-b82ab329622a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:32.652017 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc80fa35-ff2f-4a1f-8d44-b82ab329622a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:32.653202 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc80fa35-ff2f-4a1f-8d44-b82ab329622a None admin] RBAC: Adding query filter params (role.id=cc5c3091b148423684e5e966150bc6c6, user.id=77b3d29b19584ac29e46c2729c7726b0, scope.system=all) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:22:32.653663 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc80fa35-ff2f-4a1f-8d44-b82ab329622a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:32.655258 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc80fa35-ff2f-4a1f-8d44-b82ab329622a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:32.661008 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 149/307] 10.222.0.99 () {56 vars in 1308 bytes} [Thu Jul 14 15:22:32 2022] GET /identity/v3/role_assignments?user.id=77b3d29b19584ac29e46c2729c7726b0&scope.system=all&role.id=cc5c3091b148423684e5e966150bc6c6 => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:22:32.678476 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb8f951-42be-4d78-bfa5-228bff369a35 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:32.679019 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb8f951-42be-4d78-bfa5-228bff369a35 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:32.679447 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb8f951-42be-4d78-bfa5-228bff369a35 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:32.681217 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 150/308] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:22:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:22:32.689552 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6883cb-5011-41cf-8f06-18de2606751a None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:32.689850 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6883cb-5011-41cf-8f06-18de2606751a None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:32.690137 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6883cb-5011-41cf-8f06-18de2606751a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:32.712304 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-7a6883cb-5011-41cf-8f06-18de2606751a None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:22:32.713654 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-7a6883cb-5011-41cf-8f06-18de2606751a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:22:32.748328 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93da67b9-2f0c-446d-b884-8a39e721ad4a None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:32.784667 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:32.792609 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dbad8a-d5d8-4393-9760-99efc4261c67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:32.793715 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dbad8a-d5d8-4393-9760-99efc4261c67 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:32.794161 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dbad8a-d5d8-4393-9760-99efc4261c67 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:32.794611 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dbad8a-d5d8-4393-9760-99efc4261c67 None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:32.795554 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dbad8a-d5d8-4393-9760-99efc4261c67 None admin] RBAC: Adding query filter params (name=member) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:22:32.796064 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dbad8a-d5d8-4393-9760-99efc4261c67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:32.798020 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dbad8a-d5d8-4393-9760-99efc4261c67 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:32.801699 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 151/309] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:22:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:22:32.816208 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 159/310] 10.222.0.99 () {56 vars in 1101 bytes} [Thu Jul 14 15:22:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:32.821455 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6883cb-5011-41cf-8f06-18de2606751a None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:32.827348 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dbad8a-d5d8-4393-9760-99efc4261c67 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:32.956490 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6883cb-5011-41cf-8f06-18de2606751a None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:32.962869 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dbad8a-d5d8-4393-9760-99efc4261c67 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:32.972622 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:32.997352 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ab976e-f2fd-4850-b167-ff087b21ce7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:32.998420 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ab976e-f2fd-4850-b167-ff087b21ce7a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:32.998894 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ab976e-f2fd-4850-b167-ff087b21ce7a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:32.999234 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ab976e-f2fd-4850-b167-ff087b21ce7a None admin] PATH_INFO: `/v3/users/865db4991e9941c9a7f33c23c1190fa3` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:33.000536 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ab976e-f2fd-4850-b167-ff087b21ce7a None admin] RBAC: Authorizing `identity:get_user(user_id=865db4991e9941c9a7f33c23c1190fa3)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:33.002265 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ab976e-f2fd-4850-b167-ff087b21ce7a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:33.009075 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 152/311] 10.222.0.99 () {56 vars in 1177 bytes} [Thu Jul 14 15:22:32 2022] GET /identity/v3/users/865db4991e9941c9a7f33c23c1190fa3 => generated 293 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:33.010017 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ab976e-f2fd-4850-b167-ff087b21ce7a None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:33.016920 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d6bbb8-a699-41bb-b8a7-fe3e9dfa5bae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:33.017891 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d6bbb8-a699-41bb-b8a7-fe3e9dfa5bae None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:33.018184 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d6bbb8-a699-41bb-b8a7-fe3e9dfa5bae None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:33.018472 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d6bbb8-a699-41bb-b8a7-fe3e9dfa5bae None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:33.032833 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d6bbb8-a699-41bb-b8a7-fe3e9dfa5bae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:22:33.043003 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-c6d6bbb8-a699-41bb-b8a7-fe3e9dfa5bae None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 15:22:33.043003 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:22:33.043003 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:22:33.043003 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:22:33.043003 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:22:33.043003 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:22:33.043003 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:22:33.043003 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:22:33.043003 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:22:33.043003 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:22:33.043003 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:22:33.043003 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:22:33.043003 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:22:33.043003 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:22:33.043003 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:22:33.043003 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:22:33.043911 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:22:33.043911 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:22:33.043911 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:22:33.043911 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:22:33.043911 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:22:33.043911 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:22:33.043911 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:22:33.043911 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:22:33.043911 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:22:33.043911 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:22:33.043911 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:22:33.043911 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:22:33.043911 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:22:33.043911 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:22:33.043911 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:22:33.043911 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:22:33.044687 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:22:33.044687 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:22:33.044687 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:22:33.044687 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:22:33.044687 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:22:33.044687 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:22:33.044687 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:22:33.044687 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:22:33.044687 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:22:33.044687 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:22:33.044687 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 15:22:33.044687 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:22:33.044687 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 160/312] 10.222.0.99 () {56 vars in 1099 bytes} [Thu Jul 14 15:22:32 2022] GET /identity/v3/roles/reader => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:22:33.051789 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d6bbb8-a699-41bb-b8a7-fe3e9dfa5bae None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:33.118412 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:33.155974 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ab976e-f2fd-4850-b167-ff087b21ce7a None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:33.175580 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d6bbb8-a699-41bb-b8a7-fe3e9dfa5bae None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:33.193391 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f3a7c3-34d0-4406-972d-797b144b6a75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:33.194396 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f3a7c3-34d0-4406-972d-797b144b6a75 None admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:33.194868 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f3a7c3-34d0-4406-972d-797b144b6a75 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:33.195246 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f3a7c3-34d0-4406-972d-797b144b6a75 None admin] PATH_INFO: `/v3/system/users/865db4991e9941c9a7f33c23c1190fa3/roles/b8c98fddc0fd4436b577fd1d3361bc38` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:33.196864 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f3a7c3-34d0-4406-972d-797b144b6a75 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=865db4991e9941c9a7f33c23c1190fa3, role_id=b8c98fddc0fd4436b577fd1d3361bc38)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:33.198973 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f3a7c3-34d0-4406-972d-797b144b6a75 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:33.215640 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 153/313] 10.222.0.99 () {58 vars in 1334 bytes} [Thu Jul 14 15:22:33 2022] PUT /identity/v3/system/users/865db4991e9941c9a7f33c23c1190fa3/roles/b8c98fddc0fd4436b577fd1d3361bc38 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 15:22:33.226906 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d757bf9f-6720-41b6-ab8d-ee70f817c0d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:33.227674 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d757bf9f-6720-41b6-ab8d-ee70f817c0d6 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:33.227968 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d757bf9f-6720-41b6-ab8d-ee70f817c0d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:33.228293 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d757bf9f-6720-41b6-ab8d-ee70f817c0d6 None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:33.229241 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d757bf9f-6720-41b6-ab8d-ee70f817c0d6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:22:33.230186 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d757bf9f-6720-41b6-ab8d-ee70f817c0d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:33.231983 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d757bf9f-6720-41b6-ab8d-ee70f817c0d6 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:33.242545 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 161/314] 10.222.0.99 () {56 vars in 1101 bytes} [Thu Jul 14 15:22:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:22:33.249443 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f3a7c3-34d0-4406-972d-797b144b6a75 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:33.304397 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:33.368069 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f3a7c3-34d0-4406-972d-797b144b6a75 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:33.406379 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f981b158-ebbb-40f9-bfd0-072a41867e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:33.407256 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f981b158-ebbb-40f9-bfd0-072a41867e22 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:33.407653 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f981b158-ebbb-40f9-bfd0-072a41867e22 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:33.407894 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f981b158-ebbb-40f9-bfd0-072a41867e22 None admin] PATH_INFO: `/v3/users/c54065f8d99f443a8e0550a404b05514` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:33.409057 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f981b158-ebbb-40f9-bfd0-072a41867e22 None admin] RBAC: Authorizing `identity:get_user(user_id=c54065f8d99f443a8e0550a404b05514)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:33.410507 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f981b158-ebbb-40f9-bfd0-072a41867e22 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:33.412415 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 154/315] 10.222.0.99 () {56 vars in 1177 bytes} [Thu Jul 14 15:22:33 2022] GET /identity/v3/users/c54065f8d99f443a8e0550a404b05514 => generated 297 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:33.418803 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d757bf9f-6720-41b6-ab8d-ee70f817c0d6 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:33.528623 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d757bf9f-6720-41b6-ab8d-ee70f817c0d6 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:33.550462 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:33.568879 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13661ffc-a80c-4883-afa2-8bdee8204c31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:33.569464 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13661ffc-a80c-4883-afa2-8bdee8204c31 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:33.569748 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13661ffc-a80c-4883-afa2-8bdee8204c31 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:33.570106 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13661ffc-a80c-4883-afa2-8bdee8204c31 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:33.640218 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13661ffc-a80c-4883-afa2-8bdee8204c31 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:33.641679 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13661ffc-a80c-4883-afa2-8bdee8204c31 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:33.743107 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 162/316] 10.222.0.99 () {58 vars in 1282 bytes} [Thu Jul 14 15:22:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 14 15:22:33.749546 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f981b158-ebbb-40f9-bfd0-072a41867e22 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:33.864133 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f981b158-ebbb-40f9-bfd0-072a41867e22 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:33.878361 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:33.900286 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33b217c-2a48-453a-aa40-439c2b0764f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:33.901155 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33b217c-2a48-453a-aa40-439c2b0764f2 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:33.902006 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33b217c-2a48-453a-aa40-439c2b0764f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:33.902291 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33b217c-2a48-453a-aa40-439c2b0764f2 None admin] PATH_INFO: `/v3/projects/925bf9baf40d4a888a9d633d1deb5134` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:33.903500 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33b217c-2a48-453a-aa40-439c2b0764f2 None admin] RBAC: Authorizing `identity:get_project(project_id=925bf9baf40d4a888a9d633d1deb5134)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:33.904987 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33b217c-2a48-453a-aa40-439c2b0764f2 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:33.906996 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 155/317] 10.222.0.99 () {56 vars in 1186 bytes} [Thu Jul 14 15:22:33 2022] GET /identity/v3/projects/925bf9baf40d4a888a9d633d1deb5134 => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:33.912466 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13661ffc-a80c-4883-afa2-8bdee8204c31 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:33.931055 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:34.027942 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13661ffc-a80c-4883-afa2-8bdee8204c31 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:34.063740 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1671d69-d331-4d62-929a-b82c17661442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:34.064410 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1671d69-d331-4d62-929a-b82c17661442 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:34.064697 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1671d69-d331-4d62-929a-b82c17661442 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:34.064977 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1671d69-d331-4d62-929a-b82c17661442 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:34.065816 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1671d69-d331-4d62-929a-b82c17661442 None admin] RBAC: Adding query filter params (role.id=cc5c3091b148423684e5e966150bc6c6, scope.project.id=925bf9baf40d4a888a9d633d1deb5134, user.id=c54065f8d99f443a8e0550a404b05514) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:22:34.066129 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1671d69-d331-4d62-929a-b82c17661442 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:34.067568 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1671d69-d331-4d62-929a-b82c17661442 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:34.074493 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 163/318] 10.222.0.99 () {56 vars in 1374 bytes} [Thu Jul 14 15:22:33 2022] GET /identity/v3/role_assignments?user.id=c54065f8d99f443a8e0550a404b05514&scope.project.id=925bf9baf40d4a888a9d633d1deb5134&role.id=cc5c3091b148423684e5e966150bc6c6 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:34.466567 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad2a70c-94a6-48c4-aac8-098f76b92a55 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:34.466967 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad2a70c-94a6-48c4-aac8-098f76b92a55 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:34.467226 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad2a70c-94a6-48c4-aac8-098f76b92a55 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:34.468779 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 156/319] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:22:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 15:22:34.473780 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d2b776-fda9-4a9d-bfe4-541ebfbf3755 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:34.474231 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d2b776-fda9-4a9d-bfe4-541ebfbf3755 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:34.474392 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d2b776-fda9-4a9d-bfe4-541ebfbf3755 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:34.491304 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-53d2b776-fda9-4a9d-bfe4-541ebfbf3755 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:22:34.492512 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-53d2b776-fda9-4a9d-bfe4-541ebfbf3755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:22:34.561587 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 164/320] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:22:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:22:34.584364 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad2a70c-94a6-48c4-aac8-098f76b92a55 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:34.716002 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad2a70c-94a6-48c4-aac8-098f76b92a55 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:34.750927 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d750d1a-1edf-4224-9be9-0f1d17151345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:34.751869 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d750d1a-1edf-4224-9be9-0f1d17151345 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:34.751977 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d750d1a-1edf-4224-9be9-0f1d17151345 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:34.752230 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d750d1a-1edf-4224-9be9-0f1d17151345 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:34.753414 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d750d1a-1edf-4224-9be9-0f1d17151345 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:34.755114 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d750d1a-1edf-4224-9be9-0f1d17151345 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:34.756942 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 157/321] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:22:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:34.763493 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d2b776-fda9-4a9d-bfe4-541ebfbf3755 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:34.775771 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:34.807535 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a08c78-a657-4fac-a12d-9d62c9cfc9c4 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:34.808514 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a08c78-a657-4fac-a12d-9d62c9cfc9c4 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:34.809108 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a08c78-a657-4fac-a12d-9d62c9cfc9c4 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:34.810450 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 158/322] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:22:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 15:22:34.815441 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650d2397-e789-4f02-8f66-4dd8ea43e70f None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:34.815766 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650d2397-e789-4f02-8f66-4dd8ea43e70f None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:34.816054 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650d2397-e789-4f02-8f66-4dd8ea43e70f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:34.834621 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-650d2397-e789-4f02-8f66-4dd8ea43e70f None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:22:34.835868 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-650d2397-e789-4f02-8f66-4dd8ea43e70f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:22:34.872508 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d2b776-fda9-4a9d-bfe4-541ebfbf3755 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:34.884124 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:34.910527 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6253695f-cf61-4767-94d5-8724feff7dc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:34.911309 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6253695f-cf61-4767-94d5-8724feff7dc8 None admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:34.911648 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6253695f-cf61-4767-94d5-8724feff7dc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:34.911900 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6253695f-cf61-4767-94d5-8724feff7dc8 None admin] PATH_INFO: `/v3/groups` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:34.915626 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 159/323] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:22:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:22:34.915748 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6253695f-cf61-4767-94d5-8724feff7dc8 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:34.915748 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6253695f-cf61-4767-94d5-8724feff7dc8 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:34.940026 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650d2397-e789-4f02-8f66-4dd8ea43e70f None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:34.940218 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 165/324] 10.222.0.99 () {60 vars in 1134 bytes} [Thu Jul 14 15:22:34 2022] POST /identity/v3/groups => generated 226 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:22:35.062010 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650d2397-e789-4f02-8f66-4dd8ea43e70f None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:35.102314 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830896b5-1b54-4f5d-a87c-8435f521cbc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:35.103682 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830896b5-1b54-4f5d-a87c-8435f521cbc2 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:35.103682 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830896b5-1b54-4f5d-a87c-8435f521cbc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:35.103682 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830896b5-1b54-4f5d-a87c-8435f521cbc2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:35.116515 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830896b5-1b54-4f5d-a87c-8435f521cbc2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:22:35.126470 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-830896b5-1b54-4f5d-a87c-8435f521cbc2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 15:22:35.126470 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:22:35.126470 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:22:35.126470 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:22:35.126470 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:22:35.126470 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:22:35.126470 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:22:35.126470 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:22:35.126470 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:22:35.126470 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:22:35.126470 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:22:35.126470 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:22:35.126470 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:22:35.126470 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:22:35.126470 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:22:35.126470 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:22:35.127383 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:22:35.127383 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:22:35.127383 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:22:35.127383 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:22:35.127383 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:22:35.127383 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:22:35.127383 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:22:35.127383 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:22:35.127383 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:22:35.127383 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:22:35.127383 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:22:35.127383 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:22:35.127383 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:22:35.127383 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:22:35.127383 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:22:35.127383 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:22:35.128325 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:22:35.128325 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:22:35.128325 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:22:35.128325 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:22:35.128325 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:22:35.128325 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:22:35.128325 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:22:35.128325 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:22:35.128325 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:22:35.128325 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:22:35.128325 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 15:22:35.128325 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:22:35.128325 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 160/325] 10.222.0.99 () {56 vars in 1099 bytes} [Thu Jul 14 15:22:34 2022] GET /identity/v3/roles/reader => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 15:22:35.135241 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6253695f-cf61-4767-94d5-8724feff7dc8 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:35.220327 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:35.256282 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6253695f-cf61-4767-94d5-8724feff7dc8 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:35.298385 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b1ffae-64fc-42af-bdce-e5be97bcbfdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:35.299074 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b1ffae-64fc-42af-bdce-e5be97bcbfdb None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:35.299365 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b1ffae-64fc-42af-bdce-e5be97bcbfdb None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:35.299677 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b1ffae-64fc-42af-bdce-e5be97bcbfdb None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:35.300444 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b1ffae-64fc-42af-bdce-e5be97bcbfdb None admin] RBAC: Adding query filter params (name=reader) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:22:35.302343 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b1ffae-64fc-42af-bdce-e5be97bcbfdb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:35.302343 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b1ffae-64fc-42af-bdce-e5be97bcbfdb None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:35.312301 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 166/326] 10.222.0.99 () {56 vars in 1101 bytes} [Thu Jul 14 15:22:35 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:35.317791 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830896b5-1b54-4f5d-a87c-8435f521cbc2 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:35.352310 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0d1a39-e1cb-4a82-a047-e17f2be2240e None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:35.352607 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0d1a39-e1cb-4a82-a047-e17f2be2240e None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:35.352881 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0d1a39-e1cb-4a82-a047-e17f2be2240e None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:35.354526 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 167/327] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:22:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:22:35.360195 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3655f2-7cfa-4b71-a2b7-868c2ed76298 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:35.360491 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3655f2-7cfa-4b71-a2b7-868c2ed76298 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:35.360777 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3655f2-7cfa-4b71-a2b7-868c2ed76298 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:35.362438 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:35.382773 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-2a3655f2-7cfa-4b71-a2b7-868c2ed76298 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:22:35.385485 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-2a3655f2-7cfa-4b71-a2b7-868c2ed76298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:22:35.448270 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830896b5-1b54-4f5d-a87c-8435f521cbc2 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:35.490186 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 168/328] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:22:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 15:22:35.491013 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfba4d6f-046f-4d15-84a4-e5f158cbba8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:35.492535 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfba4d6f-046f-4d15-84a4-e5f158cbba8d None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:35.492535 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfba4d6f-046f-4d15-84a4-e5f158cbba8d None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:35.492535 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfba4d6f-046f-4d15-84a4-e5f158cbba8d None admin] PATH_INFO: `/v3/users/77b3d29b19584ac29e46c2729c7726b0` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:35.494129 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfba4d6f-046f-4d15-84a4-e5f158cbba8d None admin] RBAC: Authorizing `identity:get_user(user_id=77b3d29b19584ac29e46c2729c7726b0)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:35.495102 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfba4d6f-046f-4d15-84a4-e5f158cbba8d None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:35.498600 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 161/329] 10.222.0.99 () {56 vars in 1177 bytes} [Thu Jul 14 15:22:35 2022] GET /identity/v3/users/77b3d29b19584ac29e46c2729c7726b0 => generated 293 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:35.504655 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3655f2-7cfa-4b71-a2b7-868c2ed76298 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:35.515675 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfba4d6f-046f-4d15-84a4-e5f158cbba8d None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:35.595571 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:35.644456 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfba4d6f-046f-4d15-84a4-e5f158cbba8d None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:35.651152 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3655f2-7cfa-4b71-a2b7-868c2ed76298 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:35.655219 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:35.692056 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabad3bb-7e2a-4685-b1bf-4e3f56ce9632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:35.692865 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabad3bb-7e2a-4685-b1bf-4e3f56ce9632 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:35.693154 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabad3bb-7e2a-4685-b1bf-4e3f56ce9632 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:35.693433 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabad3bb-7e2a-4685-b1bf-4e3f56ce9632 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:35.707502 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabad3bb-7e2a-4685-b1bf-4e3f56ce9632 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:22:35.714914 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22b38d8-9047-4d3a-8362-71b4ef2499fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:35.715898 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22b38d8-9047-4d3a-8362-71b4ef2499fc None admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:35.716189 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22b38d8-9047-4d3a-8362-71b4ef2499fc None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:35.716472 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22b38d8-9047-4d3a-8362-71b4ef2499fc None admin] PATH_INFO: `/v3/system/users/77b3d29b19584ac29e46c2729c7726b0/roles/cc5c3091b148423684e5e966150bc6c6` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:35.717455 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-fabad3bb-7e2a-4685-b1bf-4e3f56ce9632 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 15:22:35.717455 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:22:35.717455 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:22:35.717455 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:22:35.717455 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:22:35.717455 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:22:35.717455 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:22:35.717455 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:22:35.717455 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:22:35.717455 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:22:35.717455 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:22:35.717455 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:22:35.717455 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:22:35.717455 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:22:35.717455 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:22:35.717455 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:22:35.718309 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:22:35.718309 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:22:35.718309 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:22:35.718309 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:22:35.718309 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:22:35.718309 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:22:35.718309 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:22:35.718309 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:22:35.718309 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:22:35.718309 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:22:35.718309 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:22:35.718309 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:22:35.718309 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:22:35.718309 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:22:35.718309 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:22:35.718309 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:22:35.719088 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:22:35.719088 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:22:35.719088 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:22:35.719088 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:22:35.719088 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:22:35.719088 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:22:35.719088 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:22:35.719088 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:22:35.719088 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:22:35.719088 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:22:35.719088 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 15:22:35.719088 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:22:35.720361 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22b38d8-9047-4d3a-8362-71b4ef2499fc None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=77b3d29b19584ac29e46c2729c7726b0, role_id=cc5c3091b148423684e5e966150bc6c6)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:35.721905 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22b38d8-9047-4d3a-8362-71b4ef2499fc None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:35.722143 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 162/330] 10.222.0.99 () {56 vars in 1099 bytes} [Thu Jul 14 15:22:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 15:22:35.728402 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabad3bb-7e2a-4685-b1bf-4e3f56ce9632 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:35.741778 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 169/331] 10.222.0.99 () {58 vars in 1334 bytes} [Thu Jul 14 15:22:35 2022] PUT /identity/v3/system/users/77b3d29b19584ac29e46c2729c7726b0/roles/cc5c3091b148423684e5e966150bc6c6 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:22:35.859975 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabad3bb-7e2a-4685-b1bf-4e3f56ce9632 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:35.901711 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58166619-493b-4ba2-8a58-bf403b5d87d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:35.903210 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58166619-493b-4ba2-8a58-bf403b5d87d9 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:35.903713 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58166619-493b-4ba2-8a58-bf403b5d87d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:35.904173 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58166619-493b-4ba2-8a58-bf403b5d87d9 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:35.905101 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58166619-493b-4ba2-8a58-bf403b5d87d9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:22:35.905559 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58166619-493b-4ba2-8a58-bf403b5d87d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:35.907261 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58166619-493b-4ba2-8a58-bf403b5d87d9 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:35.917765 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 163/332] 10.222.0.99 () {56 vars in 1101 bytes} [Thu Jul 14 15:22:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:35.923917 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22b38d8-9047-4d3a-8362-71b4ef2499fc None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:36.060442 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22b38d8-9047-4d3a-8362-71b4ef2499fc None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:36.088916 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:36.108529 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299411dc-cb56-47e8-bb22-1e8dca440dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:36.109435 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299411dc-cb56-47e8-bb22-1e8dca440dca None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:36.109724 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299411dc-cb56-47e8-bb22-1e8dca440dca None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:36.110065 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299411dc-cb56-47e8-bb22-1e8dca440dca None admin] PATH_INFO: `/v3/users/865db4991e9941c9a7f33c23c1190fa3` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:36.111265 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299411dc-cb56-47e8-bb22-1e8dca440dca None admin] RBAC: Authorizing `identity:get_user(user_id=865db4991e9941c9a7f33c23c1190fa3)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:36.112772 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299411dc-cb56-47e8-bb22-1e8dca440dca None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:36.114797 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 170/333] 10.222.0.99 () {56 vars in 1177 bytes} [Thu Jul 14 15:22:35 2022] GET /identity/v3/users/865db4991e9941c9a7f33c23c1190fa3 => generated 293 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:36.121063 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58166619-493b-4ba2-8a58-bf403b5d87d9 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:36.178313 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:36.281168 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58166619-493b-4ba2-8a58-bf403b5d87d9 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:36.331290 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df56a94c-c5a9-4f8f-ad29-09a42737ad53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:36.331993 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df56a94c-c5a9-4f8f-ad29-09a42737ad53 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:36.332286 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df56a94c-c5a9-4f8f-ad29-09a42737ad53 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:36.332577 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df56a94c-c5a9-4f8f-ad29-09a42737ad53 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:36.333421 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df56a94c-c5a9-4f8f-ad29-09a42737ad53 None admin] RBAC: Adding query filter params (role.id=b8c98fddc0fd4436b577fd1d3361bc38, user.id=865db4991e9941c9a7f33c23c1190fa3, scope.system=all) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:22:36.333853 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df56a94c-c5a9-4f8f-ad29-09a42737ad53 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:36.335322 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df56a94c-c5a9-4f8f-ad29-09a42737ad53 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:36.352819 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 164/334] 10.222.0.99 () {56 vars in 1308 bytes} [Thu Jul 14 15:22:36 2022] GET /identity/v3/role_assignments?user.id=865db4991e9941c9a7f33c23c1190fa3&scope.system=all&role.id=b8c98fddc0fd4436b577fd1d3361bc38 => generated 510 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:37.993024 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e4d2e5-ccb7-47bf-b398-800569d51abf None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:37.993603 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e4d2e5-ccb7-47bf-b398-800569d51abf None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:37.993603 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e4d2e5-ccb7-47bf-b398-800569d51abf None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:37.995333 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 171/335] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:22:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:22:38.001660 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8374a50b-0b88-4067-bbb8-d1c61e3972e7 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:38.002170 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8374a50b-0b88-4067-bbb8-d1c61e3972e7 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:38.002262 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8374a50b-0b88-4067-bbb8-d1c61e3972e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:38.026371 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-8374a50b-0b88-4067-bbb8-d1c61e3972e7 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:22:38.028155 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-8374a50b-0b88-4067-bbb8-d1c61e3972e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:22:38.119448 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 165/336] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:22:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:22:38.158454 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e4d2e5-ccb7-47bf-b398-800569d51abf None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:38.295279 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e4d2e5-ccb7-47bf-b398-800569d51abf None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:38.335845 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68133924-69c9-4617-8323-3ae1eea2b4af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:38.337091 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68133924-69c9-4617-8323-3ae1eea2b4af None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:38.337601 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68133924-69c9-4617-8323-3ae1eea2b4af None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:38.338090 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68133924-69c9-4617-8323-3ae1eea2b4af None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:38.354433 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68133924-69c9-4617-8323-3ae1eea2b4af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:22:38.364133 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-68133924-69c9-4617-8323-3ae1eea2b4af None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 15:22:38.364133 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:22:38.364133 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:22:38.364133 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:22:38.364133 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:22:38.364133 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:22:38.364133 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:22:38.364133 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:22:38.364133 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:22:38.364133 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:22:38.364133 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:22:38.364133 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:22:38.364133 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:22:38.364133 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:22:38.364133 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:22:38.364133 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:22:38.365228 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:22:38.365228 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:22:38.365228 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:22:38.365228 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:22:38.365228 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:22:38.365228 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:22:38.365228 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:22:38.365228 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:22:38.365228 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:22:38.365228 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:22:38.365228 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:22:38.365228 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:22:38.365228 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:22:38.365228 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:22:38.365228 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:22:38.365228 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:22:38.366150 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:22:38.366150 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:22:38.366150 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:22:38.366150 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:22:38.366150 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:22:38.366150 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:22:38.366150 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:22:38.366150 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:22:38.366150 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:22:38.366150 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:22:38.366150 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 15:22:38.366150 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:22:38.369625 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 172/337] 10.222.0.99 () {56 vars in 1114 bytes} [Thu Jul 14 15:22:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:22:38.379003 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8374a50b-0b88-4067-bbb8-d1c61e3972e7 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:38.406616 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:38.409661 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f681f6-2e82-4003-9d36-d8cbe7adb7eb None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:38.411005 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f681f6-2e82-4003-9d36-d8cbe7adb7eb None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:38.411542 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f681f6-2e82-4003-9d36-d8cbe7adb7eb None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:38.413680 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 173/338] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:22:38 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:22:38.420898 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb23243-f251-4293-bb99-215f2a2d3b2f None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:38.421568 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb23243-f251-4293-bb99-215f2a2d3b2f None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:38.422099 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb23243-f251-4293-bb99-215f2a2d3b2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:38.452716 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-8bb23243-f251-4293-bb99-215f2a2d3b2f None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:22:38.454544 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-8bb23243-f251-4293-bb99-215f2a2d3b2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:22:38.535817 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8374a50b-0b88-4067-bbb8-d1c61e3972e7 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:38.559943 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:38.590114 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cecf23-e92a-4f2f-bc5c-eca4acbf51b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:38.591002 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cecf23-e92a-4f2f-bc5c-eca4acbf51b6 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:38.591295 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cecf23-e92a-4f2f-bc5c-eca4acbf51b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:38.591576 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cecf23-e92a-4f2f-bc5c-eca4acbf51b6 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:38.592383 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cecf23-e92a-4f2f-bc5c-eca4acbf51b6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:22:38.592694 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cecf23-e92a-4f2f-bc5c-eca4acbf51b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:38.594344 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cecf23-e92a-4f2f-bc5c-eca4acbf51b6 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:38.605884 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 174/339] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:22:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 15:22:38.607315 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43094ce2-83a7-465e-bf81-5f0933d65448 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:38.607820 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43094ce2-83a7-465e-bf81-5f0933d65448 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:38.608116 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43094ce2-83a7-465e-bf81-5f0933d65448 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:38.609572 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 175/340] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:22:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:22:38.615419 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b7c081-3a7a-40bc-985f-ce63413d6bc2 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:38.615773 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b7c081-3a7a-40bc-985f-ce63413d6bc2 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:38.616065 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b7c081-3a7a-40bc-985f-ce63413d6bc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:38.617800 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 166/341] 10.222.0.99 () {56 vars in 1111 bytes} [Thu Jul 14 15:22:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:38.622495 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cecf23-e92a-4f2f-bc5c-eca4acbf51b6 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:38.651028 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-79b7c081-3a7a-40bc-985f-ce63413d6bc2 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:22:38.652353 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-79b7c081-3a7a-40bc-985f-ce63413d6bc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:22:38.758715 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:38.786988 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cecf23-e92a-4f2f-bc5c-eca4acbf51b6 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:38.792184 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 176/342] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:22:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 15:22:38.799879 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b7c081-3a7a-40bc-985f-ce63413d6bc2 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:38.838217 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5620d1b2-b765-493f-a6f8-1b7f63bd7f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:38.839224 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5620d1b2-b765-493f-a6f8-1b7f63bd7f72 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:38.840610 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5620d1b2-b765-493f-a6f8-1b7f63bd7f72 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:38.841115 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5620d1b2-b765-493f-a6f8-1b7f63bd7f72 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:38.933025 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5620d1b2-b765-493f-a6f8-1b7f63bd7f72 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:38.935363 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5620d1b2-b765-493f-a6f8-1b7f63bd7f72 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:38.950268 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b7c081-3a7a-40bc-985f-ce63413d6bc2 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:38.967465 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:39.004334 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9586396b-a78a-46e4-8f9c-21395b141f1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:39.007045 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9586396b-a78a-46e4-8f9c-21395b141f1d None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:39.008899 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9586396b-a78a-46e4-8f9c-21395b141f1d None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:39.009383 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9586396b-a78a-46e4-8f9c-21395b141f1d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:39.024014 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9586396b-a78a-46e4-8f9c-21395b141f1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:22:39.033239 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-9586396b-a78a-46e4-8f9c-21395b141f1d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 15:22:39.033239 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:22:39.033239 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:22:39.033239 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:22:39.033239 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:22:39.033239 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:22:39.033239 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:22:39.033239 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:22:39.033239 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:22:39.033239 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:22:39.033239 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:22:39.033239 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:22:39.033239 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:22:39.033239 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:22:39.033239 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:22:39.033239 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:22:39.034169 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:22:39.034169 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:22:39.034169 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:22:39.034169 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:22:39.034169 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:22:39.034169 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:22:39.034169 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:22:39.034169 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:22:39.034169 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:22:39.034169 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:22:39.034169 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:22:39.034169 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:22:39.034169 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:22:39.034169 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:22:39.034169 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:22:39.034169 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:22:39.034925 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:22:39.034925 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:22:39.034925 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:22:39.034925 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:22:39.034925 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:22:39.034925 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:22:39.034925 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:22:39.034925 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:22:39.034925 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:22:39.034925 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:22:39.034925 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 15:22:39.034925 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:22:39.037422 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:39.039020 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 177/343] 10.222.0.99 () {56 vars in 1099 bytes} [Thu Jul 14 15:22:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:22:39.039209 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79e1198-5363-4cda-9677-3b5aa1ecf17b None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:39.039618 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79e1198-5363-4cda-9677-3b5aa1ecf17b None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:39.040651 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79e1198-5363-4cda-9677-3b5aa1ecf17b None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:39.042201 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 178/344] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:22:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:22:39.048401 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79e1198-5363-4cda-9677-3b5aa1ecf17b None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:39.070441 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 167/345] 10.222.0.99 () {58 vars in 1282 bytes} [Thu Jul 14 15:22:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 15:22:39.071713 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5620d1b2-b765-493f-a6f8-1b7f63bd7f72 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:39.183311 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79e1198-5363-4cda-9677-3b5aa1ecf17b None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:39.207255 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5620d1b2-b765-493f-a6f8-1b7f63bd7f72 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:39.227564 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dd404f-e59e-40b4-9ef6-cf7969e79cf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:39.228450 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dd404f-e59e-40b4-9ef6-cf7969e79cf7 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:39.228790 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dd404f-e59e-40b4-9ef6-cf7969e79cf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:39.229188 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dd404f-e59e-40b4-9ef6-cf7969e79cf7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:39.242537 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dd404f-e59e-40b4-9ef6-cf7969e79cf7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:22:39.244707 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28263251-7c95-4d88-845b-071ecb10e920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:39.245383 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28263251-7c95-4d88-845b-071ecb10e920 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:39.245705 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28263251-7c95-4d88-845b-071ecb10e920 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:39.246087 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28263251-7c95-4d88-845b-071ecb10e920 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:39.246854 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28263251-7c95-4d88-845b-071ecb10e920 None admin] RBAC: Adding query filter params (name=member) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:22:39.247165 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28263251-7c95-4d88-845b-071ecb10e920 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:39.248575 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28263251-7c95-4d88-845b-071ecb10e920 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:39.251110 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-d6dd404f-e59e-40b4-9ef6-cf7969e79cf7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 15:22:39.251110 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:22:39.251110 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:22:39.251110 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:22:39.251110 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:22:39.251110 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:22:39.251110 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:22:39.251110 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:22:39.251110 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:22:39.251110 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:22:39.251110 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:22:39.251110 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:22:39.251110 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:22:39.251110 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:22:39.251110 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:22:39.251110 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:22:39.252514 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:22:39.252514 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:22:39.252514 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:22:39.252514 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:22:39.252514 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:22:39.252514 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:22:39.252514 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:22:39.252514 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:22:39.252514 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:22:39.252514 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:22:39.252514 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:22:39.252514 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:22:39.252514 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:22:39.252514 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:22:39.252514 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:22:39.252514 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:22:39.254938 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:22:39.254938 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:22:39.254938 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:22:39.254938 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:22:39.254938 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:22:39.254938 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:22:39.254938 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:22:39.254938 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:22:39.254938 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:22:39.254938 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:22:39.254938 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 15:22:39.254938 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:22:39.254938 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 179/346] 10.222.0.99 () {56 vars in 1099 bytes} [Thu Jul 14 15:22:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:22:39.254938 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91ae30a-19ff-4e51-b26d-2d1013af195a None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:39.257346 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91ae30a-19ff-4e51-b26d-2d1013af195a None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:39.257346 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91ae30a-19ff-4e51-b26d-2d1013af195a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:39.263771 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 168/347] 10.222.0.99 () {56 vars in 1101 bytes} [Thu Jul 14 15:22:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:39.263771 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28263251-7c95-4d88-845b-071ecb10e920 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:39.289089 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-c91ae30a-19ff-4e51-b26d-2d1013af195a None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:22:39.291274 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-c91ae30a-19ff-4e51-b26d-2d1013af195a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:22:39.312016 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:39.333681 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:39.393335 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 180/348] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:22:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:22:39.394702 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91ae30a-19ff-4e51-b26d-2d1013af195a None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:39.407900 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28263251-7c95-4d88-845b-071ecb10e920 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:39.468892 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f4fc6f-49f5-48b3-be3d-ea9d8466d13c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:39.469687 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f4fc6f-49f5-48b3-be3d-ea9d8466d13c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:39.469972 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f4fc6f-49f5-48b3-be3d-ea9d8466d13c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:39.470254 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f4fc6f-49f5-48b3-be3d-ea9d8466d13c None admin] PATH_INFO: `/v3/projects/1d9e82a46b1c4b25939e533c26cb44df` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:39.472378 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f4fc6f-49f5-48b3-be3d-ea9d8466d13c None admin] RBAC: Authorizing `identity:get_project(project_id=1d9e82a46b1c4b25939e533c26cb44df)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:39.473965 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f4fc6f-49f5-48b3-be3d-ea9d8466d13c None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:39.476350 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 169/349] 10.222.0.99 () {56 vars in 1186 bytes} [Thu Jul 14 15:22:39 2022] GET /identity/v3/projects/1d9e82a46b1c4b25939e533c26cb44df => generated 298 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:39.477804 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f4fc6f-49f5-48b3-be3d-ea9d8466d13c None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:39.565919 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91ae30a-19ff-4e51-b26d-2d1013af195a None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:39.626797 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9beefe2-3d28-403c-90c2-62590a5ce45e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:39.627553 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9beefe2-3d28-403c-90c2-62590a5ce45e None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:39.627882 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9beefe2-3d28-403c-90c2-62590a5ce45e None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:39.628167 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9beefe2-3d28-403c-90c2-62590a5ce45e None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:39.628996 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9beefe2-3d28-403c-90c2-62590a5ce45e None admin] RBAC: Adding query filter params (name=member) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:22:39.629313 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9beefe2-3d28-403c-90c2-62590a5ce45e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:39.631074 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9beefe2-3d28-403c-90c2-62590a5ce45e None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:39.641762 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 181/350] 10.222.0.99 () {56 vars in 1101 bytes} [Thu Jul 14 15:22:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:39.642288 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f4fc6f-49f5-48b3-be3d-ea9d8466d13c None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:39.642747 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9beefe2-3d28-403c-90c2-62590a5ce45e None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:39.656282 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:39.681734 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:39.691992 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04cdd4b-3cac-4171-833c-85b215529aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:39.692756 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04cdd4b-3cac-4171-833c-85b215529aab None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:39.693125 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04cdd4b-3cac-4171-833c-85b215529aab None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:39.693566 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04cdd4b-3cac-4171-833c-85b215529aab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:39.788847 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04cdd4b-3cac-4171-833c-85b215529aab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:39.789285 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9beefe2-3d28-403c-90c2-62590a5ce45e None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:39.790858 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04cdd4b-3cac-4171-833c-85b215529aab None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:39.831575 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0857893-9358-45df-8884-1b58d1a2de8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:39.832659 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0857893-9358-45df-8884-1b58d1a2de8e None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:39.832951 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0857893-9358-45df-8884-1b58d1a2de8e None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:39.833237 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0857893-9358-45df-8884-1b58d1a2de8e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:39.845703 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0857893-9358-45df-8884-1b58d1a2de8e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:22:39.853863 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-b0857893-9358-45df-8884-1b58d1a2de8e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 15:22:39.853863 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:22:39.853863 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:22:39.853863 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:22:39.853863 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:22:39.853863 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:22:39.853863 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:22:39.853863 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:22:39.853863 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:22:39.853863 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:22:39.853863 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:22:39.853863 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:22:39.853863 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:22:39.853863 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:22:39.853863 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:22:39.853863 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:22:39.854758 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:22:39.854758 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:22:39.854758 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:22:39.854758 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:22:39.854758 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:22:39.854758 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:22:39.854758 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:22:39.854758 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:22:39.854758 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:22:39.854758 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:22:39.854758 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:22:39.854758 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:22:39.854758 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:22:39.854758 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:22:39.854758 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:22:39.854758 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:22:39.856476 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:22:39.856476 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:22:39.856476 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:22:39.856476 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:22:39.856476 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:22:39.856476 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:22:39.856476 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:22:39.856476 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:22:39.856476 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:22:39.856476 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:22:39.856476 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 15:22:39.856476 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:22:39.857089 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 182/351] 10.222.0.99 () {56 vars in 1099 bytes} [Thu Jul 14 15:22:39 2022] GET /identity/v3/roles/reader => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:22:39.857637 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0857893-9358-45df-8884-1b58d1a2de8e None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:39.933048 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 170/352] 10.222.0.99 () {58 vars in 1282 bytes} [Thu Jul 14 15:22:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 15:22:39.935376 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04cdd4b-3cac-4171-833c-85b215529aab None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:39.950658 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:39.993039 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:40.024320 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0857893-9358-45df-8884-1b58d1a2de8e None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:40.071070 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfcff6f-8858-42a7-a840-328b710d17de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:40.072248 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfcff6f-8858-42a7-a840-328b710d17de None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:40.072674 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfcff6f-8858-42a7-a840-328b710d17de None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:40.073204 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfcff6f-8858-42a7-a840-328b710d17de None admin] PATH_INFO: `/v3/groups/4e08a2fcdc89418db3618b56caf282d4` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:40.081108 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04cdd4b-3cac-4171-833c-85b215529aab None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:40.081724 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfcff6f-8858-42a7-a840-328b710d17de None admin] RBAC: Authorizing `identity:get_group(group_id=4e08a2fcdc89418db3618b56caf282d4)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:40.083698 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfcff6f-8858-42a7-a840-328b710d17de None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:40.085854 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 183/353] 10.222.0.99 () {56 vars in 1180 bytes} [Thu Jul 14 15:22:39 2022] GET /identity/v3/groups/4e08a2fcdc89418db3618b56caf282d4 => generated 226 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:40.087439 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c04ffe-bdd7-4981-9200-99dffa14e3b8 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:40.087949 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c04ffe-bdd7-4981-9200-99dffa14e3b8 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:40.088419 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c04ffe-bdd7-4981-9200-99dffa14e3b8 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:40.090254 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 184/354] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:22:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:22:40.091228 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c04ffe-bdd7-4981-9200-99dffa14e3b8 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:40.129258 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937b07c0-e032-404e-aa86-a7910077efef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:40.130489 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937b07c0-e032-404e-aa86-a7910077efef None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:40.130946 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937b07c0-e032-404e-aa86-a7910077efef None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:40.131403 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937b07c0-e032-404e-aa86-a7910077efef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:40.237304 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c04ffe-bdd7-4981-9200-99dffa14e3b8 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:40.240791 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937b07c0-e032-404e-aa86-a7910077efef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:40.243015 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937b07c0-e032-404e-aa86-a7910077efef None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:40.259070 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:40.283123 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc4f877-3852-489f-8792-80af34560473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:40.284182 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc4f877-3852-489f-8792-80af34560473 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:40.284651 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc4f877-3852-489f-8792-80af34560473 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:40.285105 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc4f877-3852-489f-8792-80af34560473 None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:40.286033 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc4f877-3852-489f-8792-80af34560473 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:22:40.286347 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc4f877-3852-489f-8792-80af34560473 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:40.287781 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc4f877-3852-489f-8792-80af34560473 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:40.302027 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 185/355] 10.222.0.99 () {56 vars in 1101 bytes} [Thu Jul 14 15:22:40 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:40.303805 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc4f877-3852-489f-8792-80af34560473 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:40.323744 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:40.373004 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 171/356] 10.222.0.99 () {58 vars in 1282 bytes} [Thu Jul 14 15:22:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 15:22:40.375676 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b14eef-445a-4f82-a145-4d373ee808b0 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:40.375676 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b14eef-445a-4f82-a145-4d373ee808b0 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:40.375676 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b14eef-445a-4f82-a145-4d373ee808b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:40.391846 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-74b14eef-445a-4f82-a145-4d373ee808b0 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:22:40.393127 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-74b14eef-445a-4f82-a145-4d373ee808b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:22:40.431169 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc4f877-3852-489f-8792-80af34560473 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:40.476764 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e86e5a7-85fe-4a0c-a5f8-609d889b8043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:40.477197 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 172/357] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:22:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:22:40.478070 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e86e5a7-85fe-4a0c-a5f8-609d889b8043 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:40.478504 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e86e5a7-85fe-4a0c-a5f8-609d889b8043 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:40.478920 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e86e5a7-85fe-4a0c-a5f8-609d889b8043 None admin] PATH_INFO: `/v3/projects/1d9e82a46b1c4b25939e533c26cb44df` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:40.479178 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b14eef-445a-4f82-a145-4d373ee808b0 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:40.480797 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e86e5a7-85fe-4a0c-a5f8-609d889b8043 None admin] RBAC: Authorizing `identity:get_project(project_id=1d9e82a46b1c4b25939e533c26cb44df)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:40.483473 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e86e5a7-85fe-4a0c-a5f8-609d889b8043 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:40.485413 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 186/358] 10.222.0.99 () {56 vars in 1186 bytes} [Thu Jul 14 15:22:40 2022] GET /identity/v3/projects/1d9e82a46b1c4b25939e533c26cb44df => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:40.487328 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea949f8f-9542-48d0-87a3-9ae0228dd940 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:40.487328 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea949f8f-9542-48d0-87a3-9ae0228dd940 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:40.487328 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea949f8f-9542-48d0-87a3-9ae0228dd940 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:40.488564 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 187/359] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:22:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:22:40.489687 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea949f8f-9542-48d0-87a3-9ae0228dd940 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:40.532359 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:40.619307 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:40.621614 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b14eef-445a-4f82-a145-4d373ee808b0 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:40.649301 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea949f8f-9542-48d0-87a3-9ae0228dd940 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:40.664743 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a44cd37-56ac-4b3c-8990-c31c2b609667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:40.666227 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a44cd37-56ac-4b3c-8990-c31c2b609667 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:40.666525 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a44cd37-56ac-4b3c-8990-c31c2b609667 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:40.666835 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a44cd37-56ac-4b3c-8990-c31c2b609667 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:40.668578 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a44cd37-56ac-4b3c-8990-c31c2b609667 None admin] RBAC: Adding query filter params (group.id=4e08a2fcdc89418db3618b56caf282d4, role.id=3627b4f45c8c4f41a7def6b15f1b8225, scope.project.id=1d9e82a46b1c4b25939e533c26cb44df) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:22:40.669070 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a44cd37-56ac-4b3c-8990-c31c2b609667 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:40.670569 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a44cd37-56ac-4b3c-8990-c31c2b609667 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:40.677676 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 173/360] 10.222.0.99 () {56 vars in 1376 bytes} [Thu Jul 14 15:22:40 2022] GET /identity/v3/role_assignments?group.id=4e08a2fcdc89418db3618b56caf282d4&scope.project.id=1d9e82a46b1c4b25939e533c26cb44df&role.id=3627b4f45c8c4f41a7def6b15f1b8225 => generated 260 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:40.678515 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a44cd37-56ac-4b3c-8990-c31c2b609667 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:40.698606 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3165316a-0c33-4f74-94d8-18a0b3050adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:40.699921 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3165316a-0c33-4f74-94d8-18a0b3050adf None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:40.700214 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3165316a-0c33-4f74-94d8-18a0b3050adf None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:40.700495 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3165316a-0c33-4f74-94d8-18a0b3050adf None admin] PATH_INFO: `/v3/users/77b3d29b19584ac29e46c2729c7726b0` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:40.702064 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3165316a-0c33-4f74-94d8-18a0b3050adf None admin] RBAC: Authorizing `identity:get_user(user_id=77b3d29b19584ac29e46c2729c7726b0)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:40.704168 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3165316a-0c33-4f74-94d8-18a0b3050adf None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:40.705909 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 188/361] 10.222.0.99 () {56 vars in 1177 bytes} [Thu Jul 14 15:22:40 2022] GET /identity/v3/users/77b3d29b19584ac29e46c2729c7726b0 => generated 293 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:40.707299 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3165316a-0c33-4f74-94d8-18a0b3050adf None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:40.832800 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a44cd37-56ac-4b3c-8990-c31c2b609667 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:40.835268 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:40.859644 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3165316a-0c33-4f74-94d8-18a0b3050adf None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:40.877282 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e455b1-812c-4727-a9f3-3809079ec8fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:40.878101 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e455b1-812c-4727-a9f3-3809079ec8fe None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:40.878390 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e455b1-812c-4727-a9f3-3809079ec8fe None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:40.878674 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e455b1-812c-4727-a9f3-3809079ec8fe None admin] PATH_INFO: `/v3/projects/925bf9baf40d4a888a9d633d1deb5134` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:40.880343 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e455b1-812c-4727-a9f3-3809079ec8fe None admin] RBAC: Authorizing `identity:get_project(project_id=925bf9baf40d4a888a9d633d1deb5134)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:40.881809 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e455b1-812c-4727-a9f3-3809079ec8fe None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:40.883540 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 174/362] 10.222.0.99 () {56 vars in 1186 bytes} [Thu Jul 14 15:22:40 2022] GET /identity/v3/projects/925bf9baf40d4a888a9d633d1deb5134 => generated 302 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:40.885920 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f55753-b560-44c5-b179-a4888befd948 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:40.886219 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f55753-b560-44c5-b179-a4888befd948 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:40.886592 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f55753-b560-44c5-b179-a4888befd948 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:40.904462 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dbe33f-a3c7-4633-a199-8034b3b39341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:40.905342 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dbe33f-a3c7-4633-a199-8034b3b39341 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:40.905632 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dbe33f-a3c7-4633-a199-8034b3b39341 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:40.905912 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dbe33f-a3c7-4633-a199-8034b3b39341 None admin] PATH_INFO: `/v3/groups/4e08a2fcdc89418db3618b56caf282d4` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:40.907395 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dbe33f-a3c7-4633-a199-8034b3b39341 None admin] RBAC: Authorizing `identity:get_group(group_id=4e08a2fcdc89418db3618b56caf282d4)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:40.909421 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-54f55753-b560-44c5-b179-a4888befd948 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:22:40.909543 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dbe33f-a3c7-4633-a199-8034b3b39341 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:40.909795 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-54f55753-b560-44c5-b179-a4888befd948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:22:40.910761 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 189/363] 10.222.0.99 () {56 vars in 1180 bytes} [Thu Jul 14 15:22:40 2022] GET /identity/v3/groups/4e08a2fcdc89418db3618b56caf282d4 => generated 226 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:40.916325 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dbe33f-a3c7-4633-a199-8034b3b39341 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:40.939736 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:41.019964 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 175/364] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:22:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:22:41.020746 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f55753-b560-44c5-b179-a4888befd948 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:41.046631 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dbe33f-a3c7-4633-a199-8034b3b39341 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:41.095000 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef37958-92bc-4c88-9703-dc051bf5ac70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:41.096340 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef37958-92bc-4c88-9703-dc051bf5ac70 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:41.096831 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef37958-92bc-4c88-9703-dc051bf5ac70 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:41.097240 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef37958-92bc-4c88-9703-dc051bf5ac70 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:41.113289 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef37958-92bc-4c88-9703-dc051bf5ac70 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:22:41.122257 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-fef37958-92bc-4c88-9703-dc051bf5ac70 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 15:22:41.122257 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:22:41.122257 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:22:41.122257 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:22:41.122257 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:22:41.122257 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:22:41.122257 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:22:41.122257 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:22:41.122257 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:22:41.122257 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:22:41.122257 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:22:41.122257 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:22:41.122257 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:22:41.122257 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:22:41.122257 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:22:41.122257 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:22:41.123193 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:22:41.123193 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:22:41.123193 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:22:41.123193 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:22:41.123193 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:22:41.123193 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:22:41.123193 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:22:41.123193 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:22:41.123193 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:22:41.123193 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:22:41.123193 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:22:41.123193 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:22:41.123193 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:22:41.123193 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:22:41.123193 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:22:41.123193 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:22:41.124992 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:22:41.124992 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:22:41.124992 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:22:41.124992 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:22:41.124992 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:22:41.124992 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:22:41.124992 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:22:41.124992 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:22:41.124992 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:22:41.124992 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:22:41.124992 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 15:22:41.124992 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:22:41.124992 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 190/365] 10.222.0.99 () {56 vars in 1114 bytes} [Thu Jul 14 15:22:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:22:41.125665 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef37958-92bc-4c88-9703-dc051bf5ac70 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:41.155300 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f55753-b560-44c5-b179-a4888befd948 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:41.158063 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:41.211589 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27686ee6-5d81-4da0-b6cc-83f3bd8e6516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:41.212708 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27686ee6-5d81-4da0-b6cc-83f3bd8e6516 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:41.213000 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27686ee6-5d81-4da0-b6cc-83f3bd8e6516 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:41.213285 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27686ee6-5d81-4da0-b6cc-83f3bd8e6516 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:41.214142 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27686ee6-5d81-4da0-b6cc-83f3bd8e6516 None admin] RBAC: Adding query filter params (role.id=cc5c3091b148423684e5e966150bc6c6, user.id=77b3d29b19584ac29e46c2729c7726b0, scope.system=all) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:22:41.214459 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27686ee6-5d81-4da0-b6cc-83f3bd8e6516 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:41.217419 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27686ee6-5d81-4da0-b6cc-83f3bd8e6516 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:41.222859 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:41.237682 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 176/366] 10.222.0.99 () {56 vars in 1308 bytes} [Thu Jul 14 15:22:41 2022] GET /identity/v3/role_assignments?user.id=77b3d29b19584ac29e46c2729c7726b0&scope.system=all&role.id=cc5c3091b148423684e5e966150bc6c6 => generated 510 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:41.238826 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27686ee6-5d81-4da0-b6cc-83f3bd8e6516 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:41.267706 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef37958-92bc-4c88-9703-dc051bf5ac70 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:41.311916 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3a0b3e-68ca-4f91-a014-88d3bf27bc82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:41.312973 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3a0b3e-68ca-4f91-a014-88d3bf27bc82 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:41.313263 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3a0b3e-68ca-4f91-a014-88d3bf27bc82 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:41.313550 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3a0b3e-68ca-4f91-a014-88d3bf27bc82 None admin] PATH_INFO: `/v3/groups/4e08a2fcdc89418db3618b56caf282d4` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:41.315965 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3a0b3e-68ca-4f91-a014-88d3bf27bc82 None admin] RBAC: Authorizing `identity:get_group(group_id=4e08a2fcdc89418db3618b56caf282d4)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:41.318007 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3a0b3e-68ca-4f91-a014-88d3bf27bc82 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:41.320061 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 191/367] 10.222.0.99 () {56 vars in 1180 bytes} [Thu Jul 14 15:22:41 2022] GET /identity/v3/groups/4e08a2fcdc89418db3618b56caf282d4 => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:41.320959 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3a0b3e-68ca-4f91-a014-88d3bf27bc82 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:41.399904 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27686ee6-5d81-4da0-b6cc-83f3bd8e6516 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:41.453757 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d2de54-df55-414f-9dfd-ebd93c24f899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:41.454716 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d2de54-df55-414f-9dfd-ebd93c24f899 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:41.455012 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d2de54-df55-414f-9dfd-ebd93c24f899 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:41.455394 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d2de54-df55-414f-9dfd-ebd93c24f899 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:41.456358 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d2de54-df55-414f-9dfd-ebd93c24f899 None admin] RBAC: Adding query filter params (group.id=4e08a2fcdc89418db3618b56caf282d4, role.id=b8c98fddc0fd4436b577fd1d3361bc38, scope.project.id=1d9e82a46b1c4b25939e533c26cb44df) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:22:41.456685 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d2de54-df55-414f-9dfd-ebd93c24f899 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:41.458179 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d2de54-df55-414f-9dfd-ebd93c24f899 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:41.467167 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 177/368] 10.222.0.99 () {56 vars in 1376 bytes} [Thu Jul 14 15:22:41 2022] GET /identity/v3/role_assignments?group.id=4e08a2fcdc89418db3618b56caf282d4&scope.project.id=1d9e82a46b1c4b25939e533c26cb44df&role.id=b8c98fddc0fd4436b577fd1d3361bc38 => generated 260 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:41.468162 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d2de54-df55-414f-9dfd-ebd93c24f899 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:41.501028 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:41.501801 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3a0b3e-68ca-4f91-a014-88d3bf27bc82 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:41.555425 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c347eed3-1e11-45f6-916a-30af181a0cc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:41.557442 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c347eed3-1e11-45f6-916a-30af181a0cc5 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:41.557442 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c347eed3-1e11-45f6-916a-30af181a0cc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:41.557442 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c347eed3-1e11-45f6-916a-30af181a0cc5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:41.574516 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c347eed3-1e11-45f6-916a-30af181a0cc5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:22:41.583792 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-c347eed3-1e11-45f6-916a-30af181a0cc5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:22:41.583792 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:22:41.583792 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:22:41.583792 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:22:41.583792 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:22:41.583792 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:22:41.583792 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:22:41.583792 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:22:41.583792 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:22:41.583792 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:22:41.583792 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:22:41.583792 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:22:41.583792 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:22:41.583792 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:22:41.583792 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:22:41.583792 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:22:41.584718 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:22:41.584718 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:22:41.584718 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:22:41.584718 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:22:41.584718 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:22:41.584718 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:22:41.584718 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:22:41.584718 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:22:41.584718 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:22:41.584718 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:22:41.584718 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:22:41.584718 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:22:41.584718 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:22:41.584718 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:22:41.584718 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:22:41.584718 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:22:41.585494 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:22:41.585494 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:22:41.585494 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:22:41.585494 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:22:41.585494 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:22:41.585494 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:22:41.585494 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:22:41.585494 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:22:41.585494 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:22:41.585494 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:22:41.585494 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:22:41.585494 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:22:41.585494 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 192/369] 10.222.0.99 () {56 vars in 1096 bytes} [Thu Jul 14 15:22:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:22:41.586725 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c347eed3-1e11-45f6-916a-30af181a0cc5 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:41.598881 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:41.629041 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d2de54-df55-414f-9dfd-ebd93c24f899 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:41.677188 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e5c49a-efad-468c-87b4-45230d0877a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:41.677909 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e5c49a-efad-468c-87b4-45230d0877a9 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:41.678206 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e5c49a-efad-468c-87b4-45230d0877a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:41.678491 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e5c49a-efad-468c-87b4-45230d0877a9 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:41.679260 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e5c49a-efad-468c-87b4-45230d0877a9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:22:41.679575 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e5c49a-efad-468c-87b4-45230d0877a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:41.681854 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e5c49a-efad-468c-87b4-45230d0877a9 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:41.693879 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 178/370] 10.222.0.99 () {56 vars in 1111 bytes} [Thu Jul 14 15:22:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:41.694572 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e5c49a-efad-468c-87b4-45230d0877a9 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:41.738298 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c347eed3-1e11-45f6-916a-30af181a0cc5 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:41.792028 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c84c103-c91c-4d03-a46a-05e2779ca148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:41.793620 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c84c103-c91c-4d03-a46a-05e2779ca148 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:41.794802 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c84c103-c91c-4d03-a46a-05e2779ca148 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:41.795266 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c84c103-c91c-4d03-a46a-05e2779ca148 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:41.796785 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c84c103-c91c-4d03-a46a-05e2779ca148 None admin] RBAC: Adding query filter params (group.id=4e08a2fcdc89418db3618b56caf282d4, role.id=b8c98fddc0fd4436b577fd1d3361bc38, scope.project.id=925bf9baf40d4a888a9d633d1deb5134) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:22:41.797312 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c84c103-c91c-4d03-a46a-05e2779ca148 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:41.799795 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c84c103-c91c-4d03-a46a-05e2779ca148 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:41.805250 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:41.810687 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 193/371] 10.222.0.99 () {56 vars in 1376 bytes} [Thu Jul 14 15:22:41 2022] GET /identity/v3/role_assignments?group.id=4e08a2fcdc89418db3618b56caf282d4&scope.project.id=925bf9baf40d4a888a9d633d1deb5134&role.id=b8c98fddc0fd4436b577fd1d3361bc38 => generated 260 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:41.811315 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c84c103-c91c-4d03-a46a-05e2779ca148 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:41.849064 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e5c49a-efad-468c-87b4-45230d0877a9 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:41.927946 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bec89e6-048d-401c-bcd1-33326d93f185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:41.928630 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bec89e6-048d-401c-bcd1-33326d93f185 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:41.928914 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bec89e6-048d-401c-bcd1-33326d93f185 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:41.932333 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bec89e6-048d-401c-bcd1-33326d93f185 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:41.933201 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bec89e6-048d-401c-bcd1-33326d93f185 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:22:41.934721 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bec89e6-048d-401c-bcd1-33326d93f185 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:41.938406 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bec89e6-048d-401c-bcd1-33326d93f185 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:41.955509 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 179/372] 10.222.0.99 () {56 vars in 1099 bytes} [Thu Jul 14 15:22:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:41.961211 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bec89e6-048d-401c-bcd1-33326d93f185 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:41.962607 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:42.010521 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c84c103-c91c-4d03-a46a-05e2779ca148 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:42.053416 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889fd02b-d7c3-4451-89bd-53a6410a3812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:42.054144 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889fd02b-d7c3-4451-89bd-53a6410a3812 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:42.054383 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889fd02b-d7c3-4451-89bd-53a6410a3812 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:42.054663 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889fd02b-d7c3-4451-89bd-53a6410a3812 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:42.168882 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bec89e6-048d-401c-bcd1-33326d93f185 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:42.193426 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889fd02b-d7c3-4451-89bd-53a6410a3812 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:42.199805 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889fd02b-d7c3-4451-89bd-53a6410a3812 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:42.239961 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c934304-ae03-4d90-a7dc-1697574f275e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:42.241026 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c934304-ae03-4d90-a7dc-1697574f275e None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:42.241519 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c934304-ae03-4d90-a7dc-1697574f275e None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:42.242000 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c934304-ae03-4d90-a7dc-1697574f275e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:42.273876 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:42.287684 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:42.313859 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 194/373] 10.222.0.99 () {58 vars in 1282 bytes} [Thu Jul 14 15:22:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 503 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 15:22:42.319800 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889fd02b-d7c3-4451-89bd-53a6410a3812 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:42.352900 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c934304-ae03-4d90-a7dc-1697574f275e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:42.354737 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c934304-ae03-4d90-a7dc-1697574f275e None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:42.432194 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889fd02b-d7c3-4451-89bd-53a6410a3812 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:42.468693 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 180/374] 10.222.0.99 () {58 vars in 1282 bytes} [Thu Jul 14 15:22:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 508 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 14 15:22:42.470039 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c934304-ae03-4d90-a7dc-1697574f275e None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:42.472331 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5682f2f-3e34-4105-8b2d-f5c5e2f7a6fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:42.473184 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5682f2f-3e34-4105-8b2d-f5c5e2f7a6fa None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:42.473470 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5682f2f-3e34-4105-8b2d-f5c5e2f7a6fa None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:42.474537 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5682f2f-3e34-4105-8b2d-f5c5e2f7a6fa None admin] PATH_INFO: `/v3/projects/925bf9baf40d4a888a9d633d1deb5134` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:42.474985 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5682f2f-3e34-4105-8b2d-f5c5e2f7a6fa None admin] RBAC: Authorizing `identity:get_project(project_id=925bf9baf40d4a888a9d633d1deb5134)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:42.476459 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5682f2f-3e34-4105-8b2d-f5c5e2f7a6fa None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:42.478451 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 195/375] 10.222.0.99 () {56 vars in 1186 bytes} [Thu Jul 14 15:22:42 2022] GET /identity/v3/projects/925bf9baf40d4a888a9d633d1deb5134 => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:42.485777 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5682f2f-3e34-4105-8b2d-f5c5e2f7a6fa None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:42.528369 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:42.542578 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:42.598692 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c934304-ae03-4d90-a7dc-1697574f275e None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:42.638649 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c3fd63-2d7e-4160-ba54-976c612e5512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:42.639931 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c3fd63-2d7e-4160-ba54-976c612e5512 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:42.640450 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c3fd63-2d7e-4160-ba54-976c612e5512 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:42.640958 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c3fd63-2d7e-4160-ba54-976c612e5512 None admin] PATH_INFO: `/v3/projects/ae21968b232140b899e10f7d7d033a88` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:42.642468 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c3fd63-2d7e-4160-ba54-976c612e5512 None admin] RBAC: Authorizing `identity:get_project(project_id=ae21968b232140b899e10f7d7d033a88)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:42.644180 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c3fd63-2d7e-4160-ba54-976c612e5512 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:42.646187 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 181/376] 10.222.0.99 () {56 vars in 1186 bytes} [Thu Jul 14 15:22:42 2022] GET /identity/v3/projects/ae21968b232140b899e10f7d7d033a88 => generated 344 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:42.651947 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c3fd63-2d7e-4160-ba54-976c612e5512 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:42.674869 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5682f2f-3e34-4105-8b2d-f5c5e2f7a6fa None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:42.717869 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54da2edc-7d1e-46c8-8d59-6649de857c14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:42.718750 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54da2edc-7d1e-46c8-8d59-6649de857c14 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:42.719033 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54da2edc-7d1e-46c8-8d59-6649de857c14 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:42.719309 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54da2edc-7d1e-46c8-8d59-6649de857c14 None admin] PATH_INFO: `/v3/groups/4e08a2fcdc89418db3618b56caf282d4` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:42.720545 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54da2edc-7d1e-46c8-8d59-6649de857c14 None admin] RBAC: Authorizing `identity:get_group(group_id=4e08a2fcdc89418db3618b56caf282d4)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:42.722118 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54da2edc-7d1e-46c8-8d59-6649de857c14 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:42.724125 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 196/377] 10.222.0.99 () {56 vars in 1180 bytes} [Thu Jul 14 15:22:42 2022] GET /identity/v3/groups/4e08a2fcdc89418db3618b56caf282d4 => generated 226 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:42.736168 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54da2edc-7d1e-46c8-8d59-6649de857c14 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:42.845518 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54da2edc-7d1e-46c8-8d59-6649de857c14 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:42.847877 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:42.887400 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478e5e8d-bd68-4fe2-87e7-279e363a7ffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:42.888105 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478e5e8d-bd68-4fe2-87e7-279e363a7ffd None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:42.888391 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478e5e8d-bd68-4fe2-87e7-279e363a7ffd None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:42.888666 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478e5e8d-bd68-4fe2-87e7-279e363a7ffd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:42.889505 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478e5e8d-bd68-4fe2-87e7-279e363a7ffd None admin] RBAC: Adding query filter params (group.id=4e08a2fcdc89418db3618b56caf282d4, role.id=3627b4f45c8c4f41a7def6b15f1b8225, scope.project.id=925bf9baf40d4a888a9d633d1deb5134) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:22:42.889817 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478e5e8d-bd68-4fe2-87e7-279e363a7ffd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:42.891250 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478e5e8d-bd68-4fe2-87e7-279e363a7ffd None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:42.897899 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 197/378] 10.222.0.99 () {56 vars in 1376 bytes} [Thu Jul 14 15:22:42 2022] GET /identity/v3/role_assignments?group.id=4e08a2fcdc89418db3618b56caf282d4&scope.project.id=925bf9baf40d4a888a9d633d1deb5134&role.id=3627b4f45c8c4f41a7def6b15f1b8225 => generated 260 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:22:42.898050 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c3fd63-2d7e-4160-ba54-976c612e5512 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:42.929019 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:42.976833 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f73a28-bddb-472e-986c-24d48f22af82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:42.977726 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f73a28-bddb-472e-986c-24d48f22af82 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:42.978017 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f73a28-bddb-472e-986c-24d48f22af82 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:42.978300 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f73a28-bddb-472e-986c-24d48f22af82 None admin] PATH_INFO: `/v3/groups/9d5e500d21354956bcd54353385f28f2` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:42.998944 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f73a28-bddb-472e-986c-24d48f22af82 None admin] RBAC: Authorizing `identity:get_group(group_id=9d5e500d21354956bcd54353385f28f2)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:43.000435 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f73a28-bddb-472e-986c-24d48f22af82 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:43.002711 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 182/379] 10.222.0.99 () {56 vars in 1180 bytes} [Thu Jul 14 15:22:42 2022] GET /identity/v3/groups/9d5e500d21354956bcd54353385f28f2 => generated 229 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:22:43.013464 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478e5e8d-bd68-4fe2-87e7-279e363a7ffd None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:43.151367 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478e5e8d-bd68-4fe2-87e7-279e363a7ffd None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:43.198363 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0af6eb-c749-4845-841e-b0343576404a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:43.199033 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0af6eb-c749-4845-841e-b0343576404a None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:43.200381 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0af6eb-c749-4845-841e-b0343576404a None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:43.200381 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0af6eb-c749-4845-841e-b0343576404a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:43.201067 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0af6eb-c749-4845-841e-b0343576404a None admin] RBAC: Adding query filter params (group.id=9d5e500d21354956bcd54353385f28f2, role.id=d0f6ba1314f14d9f9c816bdb5d0c3aac, scope.project.id=ae21968b232140b899e10f7d7d033a88) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:22:43.201383 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0af6eb-c749-4845-841e-b0343576404a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:43.202820 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0af6eb-c749-4845-841e-b0343576404a None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:43.212429 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 198/380] 10.222.0.99 () {56 vars in 1376 bytes} [Thu Jul 14 15:22:43 2022] GET /identity/v3/role_assignments?group.id=9d5e500d21354956bcd54353385f28f2&scope.project.id=ae21968b232140b899e10f7d7d033a88&role.id=d0f6ba1314f14d9f9c816bdb5d0c3aac => generated 260 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:43.223079 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff48b09-e422-47b8-b623-b9478af6906e None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:43.224082 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff48b09-e422-47b8-b623-b9478af6906e None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:43.224570 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff48b09-e422-47b8-b623-b9478af6906e None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:43.230065 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 183/381] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:22:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:22:43.232587 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a2653c-d305-4c3c-aa50-fd672e46c6d9 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:43.232882 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a2653c-d305-4c3c-aa50-fd672e46c6d9 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:43.233166 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a2653c-d305-4c3c-aa50-fd672e46c6d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:43.252479 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-99a2653c-d305-4c3c-aa50-fd672e46c6d9 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:22:43.254036 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-99a2653c-d305-4c3c-aa50-fd672e46c6d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:22:43.294400 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:43.361633 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 199/382] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:22:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 15:22:43.397898 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff48b09-e422-47b8-b623-b9478af6906e None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:43.541743 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff48b09-e422-47b8-b623-b9478af6906e None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:43.594245 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb51868-3af1-4bf3-b134-dce343b59895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:43.595074 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb51868-3af1-4bf3-b134-dce343b59895 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:43.595364 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb51868-3af1-4bf3-b134-dce343b59895 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:43.595664 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb51868-3af1-4bf3-b134-dce343b59895 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:43.610978 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb51868-3af1-4bf3-b134-dce343b59895 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:22:43.619384 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-0bb51868-3af1-4bf3-b134-dce343b59895 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 15:22:43.619384 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:22:43.619384 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:22:43.619384 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:22:43.619384 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:22:43.619384 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:22:43.619384 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:22:43.619384 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:22:43.619384 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:22:43.619384 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:22:43.619384 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:22:43.619384 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:22:43.619384 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:22:43.619384 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:22:43.619384 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:22:43.619384 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:22:43.620251 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:22:43.620251 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:22:43.620251 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:22:43.620251 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:22:43.620251 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:22:43.620251 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:22:43.620251 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:22:43.620251 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:22:43.620251 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:22:43.620251 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:22:43.620251 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:22:43.620251 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:22:43.620251 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:22:43.620251 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:22:43.620251 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:22:43.620251 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:22:43.621032 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:22:43.621032 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:22:43.621032 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:22:43.621032 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:22:43.621032 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:22:43.621032 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:22:43.621032 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:22:43.621032 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:22:43.621032 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:22:43.621032 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:22:43.621032 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 15:22:43.621032 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:22:43.624312 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 184/383] 10.222.0.99 () {56 vars in 1114 bytes} [Thu Jul 14 15:22:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:22:43.629923 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a2653c-d305-4c3c-aa50-fd672e46c6d9 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:43.773723 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a2653c-d305-4c3c-aa50-fd672e46c6d9 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:43.810763 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0f1a43-c000-42fc-b1d7-941833174eae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:43.811367 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0f1a43-c000-42fc-b1d7-941833174eae None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:43.811681 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0f1a43-c000-42fc-b1d7-941833174eae None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:43.811968 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0f1a43-c000-42fc-b1d7-941833174eae None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:43.812732 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0f1a43-c000-42fc-b1d7-941833174eae None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:22:43.813044 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0f1a43-c000-42fc-b1d7-941833174eae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:43.814451 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0f1a43-c000-42fc-b1d7-941833174eae None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:43.823332 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 200/384] 10.222.0.99 () {56 vars in 1111 bytes} [Thu Jul 14 15:22:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:43.828653 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb51868-3af1-4bf3-b134-dce343b59895 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:43.887768 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:43.955059 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb51868-3af1-4bf3-b134-dce343b59895 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:43.994131 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea1e885-c6e6-4a0b-b244-969b6f0ef730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:43.995349 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea1e885-c6e6-4a0b-b244-969b6f0ef730 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:43.995871 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea1e885-c6e6-4a0b-b244-969b6f0ef730 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:43.996470 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea1e885-c6e6-4a0b-b244-969b6f0ef730 None admin] PATH_INFO: `/v3/groups/4e08a2fcdc89418db3618b56caf282d4` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:43.997935 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea1e885-c6e6-4a0b-b244-969b6f0ef730 None admin] RBAC: Authorizing `identity:get_group(group_id=4e08a2fcdc89418db3618b56caf282d4)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:43.999642 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea1e885-c6e6-4a0b-b244-969b6f0ef730 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:44.002077 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 185/385] 10.222.0.99 () {56 vars in 1180 bytes} [Thu Jul 14 15:22:43 2022] GET /identity/v3/groups/4e08a2fcdc89418db3618b56caf282d4 => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:22:44.009123 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0f1a43-c000-42fc-b1d7-941833174eae None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:44.072700 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:44.179095 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0f1a43-c000-42fc-b1d7-941833174eae None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:44.228369 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085390e4-f2d0-4f75-8c91-eeea2af2c6ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:44.229300 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085390e4-f2d0-4f75-8c91-eeea2af2c6ac None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:44.229586 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085390e4-f2d0-4f75-8c91-eeea2af2c6ac None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:44.229911 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085390e4-f2d0-4f75-8c91-eeea2af2c6ac None admin] PATH_INFO: `/v3/projects/1d9e82a46b1c4b25939e533c26cb44df` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:44.231274 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085390e4-f2d0-4f75-8c91-eeea2af2c6ac None admin] RBAC: Authorizing `identity:get_project(project_id=1d9e82a46b1c4b25939e533c26cb44df)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:44.232758 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085390e4-f2d0-4f75-8c91-eeea2af2c6ac None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:44.234778 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 201/386] 10.222.0.99 () {56 vars in 1186 bytes} [Thu Jul 14 15:22:44 2022] GET /identity/v3/projects/1d9e82a46b1c4b25939e533c26cb44df => generated 298 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:44.240271 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea1e885-c6e6-4a0b-b244-969b6f0ef730 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:44.275096 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f266485b-7aff-4ac6-9491-33ed12e4780b None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:44.275096 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f266485b-7aff-4ac6-9491-33ed12e4780b None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:44.275096 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f266485b-7aff-4ac6-9491-33ed12e4780b None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:44.275096 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 202/387] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:22:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 15:22:44.285158 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899503fe-5fd6-44b7-9c17-e09bea101d31 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:44.285867 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899503fe-5fd6-44b7-9c17-e09bea101d31 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:44.286350 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899503fe-5fd6-44b7-9c17-e09bea101d31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:44.325419 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-899503fe-5fd6-44b7-9c17-e09bea101d31 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:22:44.327004 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-899503fe-5fd6-44b7-9c17-e09bea101d31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:22:44.376190 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea1e885-c6e6-4a0b-b244-969b6f0ef730 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:44.388223 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:44.431622 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 203/388] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:22:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 15:22:44.436372 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3b2c5a-4b75-4541-a632-21014c3f7f31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:44.437295 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3b2c5a-4b75-4541-a632-21014c3f7f31 None admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:44.437581 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3b2c5a-4b75-4541-a632-21014c3f7f31 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:44.437862 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3b2c5a-4b75-4541-a632-21014c3f7f31 None admin] PATH_INFO: `/v3/projects/1d9e82a46b1c4b25939e533c26cb44df/groups/4e08a2fcdc89418db3618b56caf282d4/roles/3627b4f45c8c4f41a7def6b15f1b8225` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:44.440247 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3b2c5a-4b75-4541-a632-21014c3f7f31 None admin] RBAC: Authorizing `identity:create_grant(project_id=1d9e82a46b1c4b25939e533c26cb44df, group_id=4e08a2fcdc89418db3618b56caf282d4, role_id=3627b4f45c8c4f41a7def6b15f1b8225)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:44.442054 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3b2c5a-4b75-4541-a632-21014c3f7f31 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:44.461818 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 186/389] 10.222.0.99 () {58 vars in 1442 bytes} [Thu Jul 14 15:22:44 2022] PUT /identity/v3/projects/1d9e82a46b1c4b25939e533c26cb44df/groups/4e08a2fcdc89418db3618b56caf282d4/roles/3627b4f45c8c4f41a7def6b15f1b8225 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:22:44.488845 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1776dda2-16bb-4c4a-b642-93c1a31a0ea3 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:44.489206 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1776dda2-16bb-4c4a-b642-93c1a31a0ea3 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:44.489486 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1776dda2-16bb-4c4a-b642-93c1a31a0ea3 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:44.491122 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 204/390] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:22:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:22:44.492052 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3b2c5a-4b75-4541-a632-21014c3f7f31 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:44.501215 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c03f36d-739d-4730-995e-5a340816c00c None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:44.501846 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c03f36d-739d-4730-995e-5a340816c00c None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:44.502356 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c03f36d-739d-4730-995e-5a340816c00c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:44.524369 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-3c03f36d-739d-4730-995e-5a340816c00c None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:22:44.525943 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-3c03f36d-739d-4730-995e-5a340816c00c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:22:44.554992 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:44.613917 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3b2c5a-4b75-4541-a632-21014c3f7f31 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:44.658203 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0329b7-c4b1-4dc4-bfc7-b80238ac7909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:44.659582 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0329b7-c4b1-4dc4-bfc7-b80238ac7909 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:44.660338 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0329b7-c4b1-4dc4-bfc7-b80238ac7909 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:44.660826 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0329b7-c4b1-4dc4-bfc7-b80238ac7909 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:44.668021 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 205/391] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:22:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:22:44.677165 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0329b7-c4b1-4dc4-bfc7-b80238ac7909 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:22:44.686249 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-7f0329b7-c4b1-4dc4-bfc7-b80238ac7909 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 15:22:44.686249 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:22:44.686249 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:22:44.686249 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:22:44.686249 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:22:44.686249 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:22:44.686249 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:22:44.686249 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:22:44.686249 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:22:44.686249 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:22:44.686249 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:22:44.686249 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:22:44.686249 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:22:44.686249 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:22:44.686249 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:22:44.686249 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:22:44.687327 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:22:44.687327 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:22:44.687327 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:22:44.687327 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:22:44.687327 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:22:44.687327 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:22:44.687327 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:22:44.687327 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:22:44.687327 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:22:44.687327 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:22:44.687327 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:22:44.687327 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:22:44.687327 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:22:44.687327 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:22:44.687327 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:22:44.687327 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:22:44.688429 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:22:44.688429 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:22:44.688429 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:22:44.688429 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:22:44.688429 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:22:44.688429 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:22:44.688429 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:22:44.688429 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:22:44.688429 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:22:44.688429 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:22:44.688429 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 15:22:44.688429 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:22:44.690423 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 187/392] 10.222.0.99 () {56 vars in 1099 bytes} [Thu Jul 14 15:22:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:22:44.699046 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c03f36d-739d-4730-995e-5a340816c00c None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:44.701628 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0329b7-c4b1-4dc4-bfc7-b80238ac7909 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:44.735591 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:44.827204 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0329b7-c4b1-4dc4-bfc7-b80238ac7909 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:44.834443 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c03f36d-739d-4730-995e-5a340816c00c None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:44.863062 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea3dd2d-8081-4365-8791-528d171b5e0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:44.863895 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea3dd2d-8081-4365-8791-528d171b5e0b None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:44.866609 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea3dd2d-8081-4365-8791-528d171b5e0b None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:44.866845 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea3dd2d-8081-4365-8791-528d171b5e0b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:44.890796 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea3dd2d-8081-4365-8791-528d171b5e0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:22:44.898699 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-fea3dd2d-8081-4365-8791-528d171b5e0b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 15:22:44.898699 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:22:44.898699 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:22:44.898699 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:22:44.898699 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:22:44.898699 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:22:44.898699 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:22:44.898699 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:22:44.898699 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:22:44.898699 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:22:44.898699 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:22:44.898699 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:22:44.898699 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:22:44.898699 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:22:44.898699 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:22:44.898699 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:22:44.899557 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:22:44.899557 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:22:44.899557 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:22:44.899557 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:22:44.899557 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:22:44.899557 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:22:44.899557 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:22:44.899557 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:22:44.899557 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:22:44.899557 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:22:44.899557 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:22:44.899557 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:22:44.899557 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:22:44.899557 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:22:44.899557 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:22:44.899557 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:22:44.900363 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:22:44.900363 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:22:44.900363 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:22:44.900363 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:22:44.900363 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:22:44.900363 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:22:44.900363 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:22:44.900363 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:22:44.900363 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:22:44.900363 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:22:44.900363 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 15:22:44.900363 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:22:44.901999 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 188/393] 10.222.0.99 () {56 vars in 1099 bytes} [Thu Jul 14 15:22:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 15:22:44.909875 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d90fcb0-4022-4062-9e89-466b1a164a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:44.910573 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d90fcb0-4022-4062-9e89-466b1a164a6a None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:44.910865 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d90fcb0-4022-4062-9e89-466b1a164a6a None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:44.911148 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d90fcb0-4022-4062-9e89-466b1a164a6a None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:44.911942 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d90fcb0-4022-4062-9e89-466b1a164a6a None admin] RBAC: Adding query filter params (name=member) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:22:44.912257 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d90fcb0-4022-4062-9e89-466b1a164a6a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:44.913646 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d90fcb0-4022-4062-9e89-466b1a164a6a None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:44.915915 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea3dd2d-8081-4365-8791-528d171b5e0b None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:44.925183 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 206/394] 10.222.0.99 () {56 vars in 1101 bytes} [Thu Jul 14 15:22:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:44.931408 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d90fcb0-4022-4062-9e89-466b1a164a6a None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:44.956474 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:45.115977 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:45.124169 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d90fcb0-4022-4062-9e89-466b1a164a6a None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:45.136721 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea3dd2d-8081-4365-8791-528d171b5e0b None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:45.192914 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59513f83-2145-4635-beba-15eb4a1054b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:45.193795 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59513f83-2145-4635-beba-15eb4a1054b8 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:45.194081 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59513f83-2145-4635-beba-15eb4a1054b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:45.194360 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59513f83-2145-4635-beba-15eb4a1054b8 None admin] PATH_INFO: `/v3/groups/4e08a2fcdc89418db3618b56caf282d4` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:45.196799 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59513f83-2145-4635-beba-15eb4a1054b8 None admin] RBAC: Authorizing `identity:get_group(group_id=4e08a2fcdc89418db3618b56caf282d4)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:45.198276 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59513f83-2145-4635-beba-15eb4a1054b8 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:45.200830 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 207/395] 10.222.0.99 () {56 vars in 1180 bytes} [Thu Jul 14 15:22:44 2022] GET /identity/v3/groups/4e08a2fcdc89418db3618b56caf282d4 => generated 226 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:45.201338 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf9fb1c-edf8-4dc0-95e3-3ce88ef925c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:45.202010 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf9fb1c-edf8-4dc0-95e3-3ce88ef925c8 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:45.202298 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf9fb1c-edf8-4dc0-95e3-3ce88ef925c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:45.202578 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf9fb1c-edf8-4dc0-95e3-3ce88ef925c8 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:45.203342 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf9fb1c-edf8-4dc0-95e3-3ce88ef925c8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:22:45.203681 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf9fb1c-edf8-4dc0-95e3-3ce88ef925c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:45.205059 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf9fb1c-edf8-4dc0-95e3-3ce88ef925c8 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:45.211897 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59513f83-2145-4635-beba-15eb4a1054b8 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:45.231971 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 189/396] 10.222.0.99 () {56 vars in 1101 bytes} [Thu Jul 14 15:22:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:45.234487 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf9fb1c-edf8-4dc0-95e3-3ce88ef925c8 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:45.320341 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:45.343835 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59513f83-2145-4635-beba-15eb4a1054b8 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:45.416601 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026878c0-9ed3-4977-90a0-02038b3c4f63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:45.417445 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026878c0-9ed3-4977-90a0-02038b3c4f63 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:45.417737 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026878c0-9ed3-4977-90a0-02038b3c4f63 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:45.418024 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026878c0-9ed3-4977-90a0-02038b3c4f63 None admin] PATH_INFO: `/v3/projects/1d9e82a46b1c4b25939e533c26cb44df` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:45.421195 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf9fb1c-edf8-4dc0-95e3-3ce88ef925c8 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:45.424077 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026878c0-9ed3-4977-90a0-02038b3c4f63 None admin] RBAC: Authorizing `identity:get_project(project_id=1d9e82a46b1c4b25939e533c26cb44df)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:45.425698 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026878c0-9ed3-4977-90a0-02038b3c4f63 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:45.427570 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 208/397] 10.222.0.99 () {56 vars in 1186 bytes} [Thu Jul 14 15:22:45 2022] GET /identity/v3/projects/1d9e82a46b1c4b25939e533c26cb44df => generated 298 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:45.433433 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026878c0-9ed3-4977-90a0-02038b3c4f63 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:45.459385 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99924cd9-b18a-4c2e-a5c0-a4bc766210ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:45.460312 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99924cd9-b18a-4c2e-a5c0-a4bc766210ce None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:45.460596 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99924cd9-b18a-4c2e-a5c0-a4bc766210ce None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:45.460874 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99924cd9-b18a-4c2e-a5c0-a4bc766210ce None admin] PATH_INFO: `/v3/groups/4e08a2fcdc89418db3618b56caf282d4` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:45.462785 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99924cd9-b18a-4c2e-a5c0-a4bc766210ce None admin] RBAC: Authorizing `identity:get_group(group_id=4e08a2fcdc89418db3618b56caf282d4)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:45.464282 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99924cd9-b18a-4c2e-a5c0-a4bc766210ce None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:45.467856 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 190/398] 10.222.0.99 () {56 vars in 1180 bytes} [Thu Jul 14 15:22:45 2022] GET /identity/v3/groups/4e08a2fcdc89418db3618b56caf282d4 => generated 226 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:45.468233 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c31834-b1b8-41c9-bebe-c16de52262c2 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:45.468527 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c31834-b1b8-41c9-bebe-c16de52262c2 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:45.468803 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c31834-b1b8-41c9-bebe-c16de52262c2 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:45.473635 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 191/399] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:22:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:22:45.475590 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419acef8-209a-42b2-94e1-330bb561c7dd None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:45.475904 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419acef8-209a-42b2-94e1-330bb561c7dd None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:45.476187 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419acef8-209a-42b2-94e1-330bb561c7dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:45.493942 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-419acef8-209a-42b2-94e1-330bb561c7dd None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:22:45.495154 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-419acef8-209a-42b2-94e1-330bb561c7dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:22:45.507258 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:45.572328 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 192/400] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:22:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 15:22:45.573559 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419acef8-209a-42b2-94e1-330bb561c7dd None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:45.582320 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026878c0-9ed3-4977-90a0-02038b3c4f63 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:45.640973 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bdaa62-5d08-4d63-a374-abc329c08663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:45.642249 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bdaa62-5d08-4d63-a374-abc329c08663 None admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:45.642604 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bdaa62-5d08-4d63-a374-abc329c08663 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:45.642956 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bdaa62-5d08-4d63-a374-abc329c08663 None admin] PATH_INFO: `/v3/projects/1d9e82a46b1c4b25939e533c26cb44df/groups/4e08a2fcdc89418db3618b56caf282d4/roles/b8c98fddc0fd4436b577fd1d3361bc38` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:45.645663 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bdaa62-5d08-4d63-a374-abc329c08663 None admin] RBAC: Authorizing `identity:create_grant(project_id=1d9e82a46b1c4b25939e533c26cb44df, group_id=4e08a2fcdc89418db3618b56caf282d4, role_id=b8c98fddc0fd4436b577fd1d3361bc38)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:45.647195 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bdaa62-5d08-4d63-a374-abc329c08663 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:45.665317 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 209/401] 10.222.0.99 () {58 vars in 1442 bytes} [Thu Jul 14 15:22:45 2022] PUT /identity/v3/projects/1d9e82a46b1c4b25939e533c26cb44df/groups/4e08a2fcdc89418db3618b56caf282d4/roles/b8c98fddc0fd4436b577fd1d3361bc38 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 15:22:45.666700 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bdaa62-5d08-4d63-a374-abc329c08663 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:45.694882 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:45.745809 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419acef8-209a-42b2-94e1-330bb561c7dd None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:45.827001 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03193c9-f5d6-49a2-a203-c5a571fc3987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:45.827916 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03193c9-f5d6-49a2-a203-c5a571fc3987 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:45.828198 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03193c9-f5d6-49a2-a203-c5a571fc3987 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:45.828471 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03193c9-f5d6-49a2-a203-c5a571fc3987 None admin] PATH_INFO: `/v3/projects/925bf9baf40d4a888a9d633d1deb5134` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:45.830945 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03193c9-f5d6-49a2-a203-c5a571fc3987 None admin] RBAC: Authorizing `identity:get_project(project_id=925bf9baf40d4a888a9d633d1deb5134)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:45.833910 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03193c9-f5d6-49a2-a203-c5a571fc3987 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:45.845341 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 193/402] 10.222.0.99 () {56 vars in 1186 bytes} [Thu Jul 14 15:22:45 2022] GET /identity/v3/projects/925bf9baf40d4a888a9d633d1deb5134 => generated 302 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:45.848378 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03193c9-f5d6-49a2-a203-c5a571fc3987 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:45.873416 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bdaa62-5d08-4d63-a374-abc329c08663 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:45.885020 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:45.923544 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75e53ae-ad5c-4017-a88c-ea9fef64dd9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:45.924742 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75e53ae-ad5c-4017-a88c-ea9fef64dd9f None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:45.925067 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75e53ae-ad5c-4017-a88c-ea9fef64dd9f None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:45.925322 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75e53ae-ad5c-4017-a88c-ea9fef64dd9f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:45.945656 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75e53ae-ad5c-4017-a88c-ea9fef64dd9f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:22:45.956875 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-c75e53ae-ad5c-4017-a88c-ea9fef64dd9f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 15:22:45.956875 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:22:45.956875 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:22:45.956875 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:22:45.956875 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:22:45.956875 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:22:45.956875 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:22:45.956875 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:22:45.956875 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:22:45.956875 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:22:45.956875 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:22:45.956875 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:22:45.956875 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:22:45.956875 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:22:45.956875 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:22:45.956875 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:22:45.957822 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:22:45.957822 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:22:45.957822 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:22:45.957822 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:22:45.957822 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:22:45.957822 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:22:45.957822 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:22:45.957822 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:22:45.957822 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:22:45.957822 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:22:45.957822 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:22:45.957822 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:22:45.957822 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:22:45.957822 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:22:45.957822 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:22:45.957822 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:22:45.958646 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:22:45.958646 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:22:45.958646 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:22:45.958646 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:22:45.958646 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:22:45.958646 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:22:45.958646 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:22:45.958646 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:22:45.958646 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:22:45.958646 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:22:45.958646 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 15:22:45.958646 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:22:45.958646 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 210/403] 10.222.0.99 () {56 vars in 1114 bytes} [Thu Jul 14 15:22:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:22:45.970467 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75e53ae-ad5c-4017-a88c-ea9fef64dd9f None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:46.006969 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03193c9-f5d6-49a2-a203-c5a571fc3987 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:46.076221 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84583f9c-dfc5-4e9f-a5ef-9a17dc709264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:46.077137 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84583f9c-dfc5-4e9f-a5ef-9a17dc709264 None admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:46.077427 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84583f9c-dfc5-4e9f-a5ef-9a17dc709264 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:46.078020 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84583f9c-dfc5-4e9f-a5ef-9a17dc709264 None admin] PATH_INFO: `/v3/projects/925bf9baf40d4a888a9d633d1deb5134/groups/4e08a2fcdc89418db3618b56caf282d4/roles/b8c98fddc0fd4436b577fd1d3361bc38` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:46.080323 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84583f9c-dfc5-4e9f-a5ef-9a17dc709264 None admin] RBAC: Authorizing `identity:create_grant(project_id=925bf9baf40d4a888a9d633d1deb5134, group_id=4e08a2fcdc89418db3618b56caf282d4, role_id=b8c98fddc0fd4436b577fd1d3361bc38)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:46.082087 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84583f9c-dfc5-4e9f-a5ef-9a17dc709264 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:46.101362 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:46.106925 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 194/404] 10.222.0.99 () {58 vars in 1442 bytes} [Thu Jul 14 15:22:45 2022] PUT /identity/v3/projects/925bf9baf40d4a888a9d633d1deb5134/groups/4e08a2fcdc89418db3618b56caf282d4/roles/b8c98fddc0fd4436b577fd1d3361bc38 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:22:46.109124 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1b37f3-89d1-4231-9f83-88de8e1cbfb4 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:46.109640 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1b37f3-89d1-4231-9f83-88de8e1cbfb4 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:46.110140 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1b37f3-89d1-4231-9f83-88de8e1cbfb4 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:46.113027 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 195/405] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:22:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:22:46.119277 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38b4e66-ea6d-4639-b37c-13bd0187d9eb None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:46.119639 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38b4e66-ea6d-4639-b37c-13bd0187d9eb None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:46.120383 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38b4e66-ea6d-4639-b37c-13bd0187d9eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:46.141974 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-c38b4e66-ea6d-4639-b37c-13bd0187d9eb None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:22:46.144287 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c38b4e66-ea6d-4639-b37c-13bd0187d9eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:22:46.151904 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75e53ae-ad5c-4017-a88c-ea9fef64dd9f None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:46.203688 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98a0f73-22e8-41d9-abfc-98884cb05f97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:46.206488 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98a0f73-22e8-41d9-abfc-98884cb05f97 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:46.207144 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98a0f73-22e8-41d9-abfc-98884cb05f97 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:46.207681 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98a0f73-22e8-41d9-abfc-98884cb05f97 None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:46.209043 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98a0f73-22e8-41d9-abfc-98884cb05f97 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:22:46.209624 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98a0f73-22e8-41d9-abfc-98884cb05f97 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:46.212632 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98a0f73-22e8-41d9-abfc-98884cb05f97 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:46.224972 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 211/406] 10.222.0.99 () {56 vars in 1111 bytes} [Thu Jul 14 15:22:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:22:46.235254 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98a0f73-22e8-41d9-abfc-98884cb05f97 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:46.258575 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 196/407] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:22:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 15:22:46.297326 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38b4e66-ea6d-4639-b37c-13bd0187d9eb None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:46.306075 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:46.374560 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98a0f73-22e8-41d9-abfc-98884cb05f97 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:46.442828 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38b4e66-ea6d-4639-b37c-13bd0187d9eb None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:46.452323 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:46.473518 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b48d135-858d-49c7-8708-fa28bc5f2feb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:46.474311 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b48d135-858d-49c7-8708-fa28bc5f2feb None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:46.474595 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b48d135-858d-49c7-8708-fa28bc5f2feb None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:46.474871 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b48d135-858d-49c7-8708-fa28bc5f2feb None admin] PATH_INFO: `/v3/groups/4e08a2fcdc89418db3618b56caf282d4` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:46.480426 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b48d135-858d-49c7-8708-fa28bc5f2feb None admin] RBAC: Authorizing `identity:get_group(group_id=4e08a2fcdc89418db3618b56caf282d4)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:46.482287 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b48d135-858d-49c7-8708-fa28bc5f2feb None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:46.485343 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 212/408] 10.222.0.99 () {56 vars in 1180 bytes} [Thu Jul 14 15:22:46 2022] GET /identity/v3/groups/4e08a2fcdc89418db3618b56caf282d4 => generated 226 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:46.489880 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b48d135-858d-49c7-8708-fa28bc5f2feb None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:46.490851 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd9683d-1399-40c1-9ed0-d3fe2fc56714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:46.491807 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd9683d-1399-40c1-9ed0-d3fe2fc56714 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:46.492168 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd9683d-1399-40c1-9ed0-d3fe2fc56714 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:46.492519 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd9683d-1399-40c1-9ed0-d3fe2fc56714 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:46.505964 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd9683d-1399-40c1-9ed0-d3fe2fc56714 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:22:46.514454 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-9fd9683d-1399-40c1-9ed0-d3fe2fc56714 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:22:46.514454 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:22:46.514454 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:22:46.514454 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:22:46.514454 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:22:46.514454 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:22:46.514454 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:22:46.514454 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:22:46.514454 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:22:46.514454 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:22:46.514454 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:22:46.514454 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:22:46.514454 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:22:46.514454 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:22:46.514454 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:22:46.514454 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:22:46.515316 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:22:46.515316 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:22:46.515316 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:22:46.515316 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:22:46.515316 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:22:46.515316 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:22:46.515316 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:22:46.515316 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:22:46.515316 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:22:46.515316 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:22:46.515316 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:22:46.515316 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:22:46.515316 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:22:46.515316 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:22:46.515316 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:22:46.515316 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:22:46.516131 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:22:46.516131 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:22:46.516131 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:22:46.516131 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:22:46.516131 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:22:46.516131 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:22:46.516131 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:22:46.516131 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:22:46.516131 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:22:46.516131 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:22:46.516131 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:22:46.516131 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:22:46.516790 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 197/409] 10.222.0.99 () {56 vars in 1096 bytes} [Thu Jul 14 15:22:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:22:46.525931 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd9683d-1399-40c1-9ed0-d3fe2fc56714 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:46.644026 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:46.665629 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b48d135-858d-49c7-8708-fa28bc5f2feb None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:46.725515 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba2ab72-03fd-47b9-8fc8-469a758e9fa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:46.726307 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba2ab72-03fd-47b9-8fc8-469a758e9fa7 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:46.726591 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba2ab72-03fd-47b9-8fc8-469a758e9fa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:46.726868 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba2ab72-03fd-47b9-8fc8-469a758e9fa7 None admin] PATH_INFO: `/v3/projects/925bf9baf40d4a888a9d633d1deb5134` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:46.731494 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba2ab72-03fd-47b9-8fc8-469a758e9fa7 None admin] RBAC: Authorizing `identity:get_project(project_id=925bf9baf40d4a888a9d633d1deb5134)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:46.733061 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba2ab72-03fd-47b9-8fc8-469a758e9fa7 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:46.734967 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 213/410] 10.222.0.99 () {56 vars in 1186 bytes} [Thu Jul 14 15:22:46 2022] GET /identity/v3/projects/925bf9baf40d4a888a9d633d1deb5134 => generated 302 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:22:46.737152 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd9683d-1399-40c1-9ed0-d3fe2fc56714 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:46.740402 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba2ab72-03fd-47b9-8fc8-469a758e9fa7 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:46.824237 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1489527b-ee97-4e6b-be0d-4d3965ced187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:46.825741 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1489527b-ee97-4e6b-be0d-4d3965ced187 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:46.826134 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1489527b-ee97-4e6b-be0d-4d3965ced187 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:46.826417 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1489527b-ee97-4e6b-be0d-4d3965ced187 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:46.827181 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1489527b-ee97-4e6b-be0d-4d3965ced187 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:22:46.827490 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1489527b-ee97-4e6b-be0d-4d3965ced187 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:46.828912 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1489527b-ee97-4e6b-be0d-4d3965ced187 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:46.847036 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:46.850612 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 198/411] 10.222.0.99 () {56 vars in 1099 bytes} [Thu Jul 14 15:22:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:46.856281 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1489527b-ee97-4e6b-be0d-4d3965ced187 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:46.937560 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba2ab72-03fd-47b9-8fc8-469a758e9fa7 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:46.982853 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1489527b-ee97-4e6b-be0d-4d3965ced187 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:47.007337 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea76ded-d5a5-4a40-b12f-b0748ea9ac2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:47.008275 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea76ded-d5a5-4a40-b12f-b0748ea9ac2f None admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:47.008567 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea76ded-d5a5-4a40-b12f-b0748ea9ac2f None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:47.008855 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea76ded-d5a5-4a40-b12f-b0748ea9ac2f None admin] PATH_INFO: `/v3/projects/925bf9baf40d4a888a9d633d1deb5134/groups/4e08a2fcdc89418db3618b56caf282d4/roles/3627b4f45c8c4f41a7def6b15f1b8225` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:47.010397 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:47.011117 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea76ded-d5a5-4a40-b12f-b0748ea9ac2f None admin] RBAC: Authorizing `identity:create_grant(project_id=925bf9baf40d4a888a9d633d1deb5134, group_id=4e08a2fcdc89418db3618b56caf282d4, role_id=3627b4f45c8c4f41a7def6b15f1b8225)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:47.012883 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea76ded-d5a5-4a40-b12f-b0748ea9ac2f None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:47.021176 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65441237-0471-4f4c-8ee1-b35f770b20a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:47.022272 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65441237-0471-4f4c-8ee1-b35f770b20a9 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:47.022697 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65441237-0471-4f4c-8ee1-b35f770b20a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:47.023115 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65441237-0471-4f4c-8ee1-b35f770b20a9 None admin] PATH_INFO: `/v3/groups/9d5e500d21354956bcd54353385f28f2` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:47.025895 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65441237-0471-4f4c-8ee1-b35f770b20a9 None admin] RBAC: Authorizing `identity:get_group(group_id=9d5e500d21354956bcd54353385f28f2)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:47.028070 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65441237-0471-4f4c-8ee1-b35f770b20a9 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:47.030541 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 199/412] 10.222.0.99 () {56 vars in 1180 bytes} [Thu Jul 14 15:22:46 2022] GET /identity/v3/groups/9d5e500d21354956bcd54353385f28f2 => generated 229 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:47.036141 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65441237-0471-4f4c-8ee1-b35f770b20a9 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:47.054575 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 214/413] 10.222.0.99 () {58 vars in 1442 bytes} [Thu Jul 14 15:22:46 2022] PUT /identity/v3/projects/925bf9baf40d4a888a9d633d1deb5134/groups/4e08a2fcdc89418db3618b56caf282d4/roles/3627b4f45c8c4f41a7def6b15f1b8225 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:22:47.170031 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f78297-a2a0-4721-8dd6-2283be6e91fc None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:47.170360 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f78297-a2a0-4721-8dd6-2283be6e91fc None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:47.170604 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f78297-a2a0-4721-8dd6-2283be6e91fc None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:47.172587 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 215/414] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:22:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:22:47.180097 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872e24e6-1b40-4894-aae9-136324fc9fb4 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:47.180413 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872e24e6-1b40-4894-aae9-136324fc9fb4 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:47.180667 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872e24e6-1b40-4894-aae9-136324fc9fb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:47.202385 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65441237-0471-4f4c-8ee1-b35f770b20a9 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:47.203519 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-872e24e6-1b40-4894-aae9-136324fc9fb4 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:22:47.206828 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-872e24e6-1b40-4894-aae9-136324fc9fb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:22:47.288492 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7e7781-244a-415c-8cdd-9e96861e4b2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:47.289722 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7e7781-244a-415c-8cdd-9e96861e4b2a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:47.290243 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7e7781-244a-415c-8cdd-9e96861e4b2a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:47.290736 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7e7781-244a-415c-8cdd-9e96861e4b2a None admin] PATH_INFO: `/v3/projects/ae21968b232140b899e10f7d7d033a88` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:47.291664 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 216/415] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:22:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 15:22:47.292484 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7e7781-244a-415c-8cdd-9e96861e4b2a None admin] RBAC: Authorizing `identity:get_project(project_id=ae21968b232140b899e10f7d7d033a88)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:47.294654 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7e7781-244a-415c-8cdd-9e96861e4b2a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:47.296894 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 200/416] 10.222.0.99 () {56 vars in 1186 bytes} [Thu Jul 14 15:22:47 2022] GET /identity/v3/projects/ae21968b232140b899e10f7d7d033a88 => generated 344 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:47.302707 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872e24e6-1b40-4894-aae9-136324fc9fb4 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:47.312227 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7e7781-244a-415c-8cdd-9e96861e4b2a None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:47.383169 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:47.385797 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:47.444338 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7e7781-244a-415c-8cdd-9e96861e4b2a None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:47.446116 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872e24e6-1b40-4894-aae9-136324fc9fb4 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:47.499839 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b29262-4821-4ece-a8c5-73e8a2903060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:47.501161 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b29262-4821-4ece-a8c5-73e8a2903060 None admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:47.501599 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b29262-4821-4ece-a8c5-73e8a2903060 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:47.502026 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b29262-4821-4ece-a8c5-73e8a2903060 None admin] PATH_INFO: `/v3/projects/ae21968b232140b899e10f7d7d033a88/groups/9d5e500d21354956bcd54353385f28f2/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:47.504262 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b29262-4821-4ece-a8c5-73e8a2903060 None admin] RBAC: Authorizing `identity:create_grant(project_id=ae21968b232140b899e10f7d7d033a88, group_id=9d5e500d21354956bcd54353385f28f2, role_id=d0f6ba1314f14d9f9c816bdb5d0c3aac)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:47.506109 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b29262-4821-4ece-a8c5-73e8a2903060 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:47.509085 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9341562f-71e5-482f-b0d7-87b36154b489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:47.510119 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9341562f-71e5-482f-b0d7-87b36154b489 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:47.510573 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9341562f-71e5-482f-b0d7-87b36154b489 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:47.510938 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9341562f-71e5-482f-b0d7-87b36154b489 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:47.522075 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 217/417] 10.222.0.99 () {58 vars in 1442 bytes} [Thu Jul 14 15:22:47 2022] PUT /identity/v3/projects/ae21968b232140b899e10f7d7d033a88/groups/9d5e500d21354956bcd54353385f28f2/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 15:22:47.547456 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9341562f-71e5-482f-b0d7-87b36154b489 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:22:47.564138 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-9341562f-71e5-482f-b0d7-87b36154b489 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 15:22:47.564138 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:22:47.564138 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:22:47.564138 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:22:47.564138 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:22:47.564138 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:22:47.564138 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:22:47.564138 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:22:47.564138 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:22:47.564138 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:22:47.564138 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:22:47.564138 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:22:47.564138 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:22:47.564138 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:22:47.564138 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:22:47.564138 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:22:47.565102 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:22:47.565102 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:22:47.565102 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:22:47.565102 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:22:47.565102 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:22:47.565102 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:22:47.565102 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:22:47.565102 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:22:47.565102 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:22:47.565102 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:22:47.565102 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:22:47.565102 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:22:47.565102 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:22:47.565102 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:22:47.565102 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:22:47.565102 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:22:47.565893 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:22:47.565893 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:22:47.565893 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:22:47.565893 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:22:47.565893 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:22:47.565893 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:22:47.565893 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:22:47.565893 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:22:47.565893 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:22:47.565893 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:22:47.565893 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 15:22:47.565893 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:22:47.565893 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 201/418] 10.222.0.99 () {56 vars in 1114 bytes} [Thu Jul 14 15:22:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 15:22:47.572365 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b29262-4821-4ece-a8c5-73e8a2903060 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:47.703511 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b29262-4821-4ece-a8c5-73e8a2903060 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:47.725682 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:47.744566 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2754c4-b071-4312-88a7-77ec7cf1219b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:47.745950 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2754c4-b071-4312-88a7-77ec7cf1219b None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:47.746243 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2754c4-b071-4312-88a7-77ec7cf1219b None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:47.747051 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2754c4-b071-4312-88a7-77ec7cf1219b None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:47.747858 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2754c4-b071-4312-88a7-77ec7cf1219b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:22:47.748173 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2754c4-b071-4312-88a7-77ec7cf1219b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:47.749566 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2754c4-b071-4312-88a7-77ec7cf1219b None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:47.758664 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 218/419] 10.222.0.99 () {56 vars in 1111 bytes} [Thu Jul 14 15:22:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:47.765204 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9341562f-71e5-482f-b0d7-87b36154b489 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:47.899977 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9341562f-71e5-482f-b0d7-87b36154b489 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:47.933602 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:47.953224 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1341f511-f6f4-4732-af44-0250ed9de92c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:47.953899 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1341f511-f6f4-4732-af44-0250ed9de92c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:47.954191 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1341f511-f6f4-4732-af44-0250ed9de92c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:47.954475 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1341f511-f6f4-4732-af44-0250ed9de92c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:48.045781 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1341f511-f6f4-4732-af44-0250ed9de92c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:48.047491 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1341f511-f6f4-4732-af44-0250ed9de92c None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:48.186148 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 202/420] 10.222.0.99 () {58 vars in 1282 bytes} [Thu Jul 14 15:22:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 14 15:22:48.191493 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2754c4-b071-4312-88a7-77ec7cf1219b None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:48.344400 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2754c4-b071-4312-88a7-77ec7cf1219b None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:48.365245 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd81b332-cb0b-4592-83ba-24c32cce3ead None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:48.365537 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd81b332-cb0b-4592-83ba-24c32cce3ead None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:48.365826 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd81b332-cb0b-4592-83ba-24c32cce3ead None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:48.367208 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 203/421] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:22:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:22:48.372957 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33892c16-3cb3-4aa8-8dcb-5c48c243bb6f None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:48.373401 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33892c16-3cb3-4aa8-8dcb-5c48c243bb6f None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:48.373687 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33892c16-3cb3-4aa8-8dcb-5c48c243bb6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:48.384118 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b49e04-7a02-44ed-8cfb-9678debfa9e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:48.385292 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b49e04-7a02-44ed-8cfb-9678debfa9e7 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:48.385778 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b49e04-7a02-44ed-8cfb-9678debfa9e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:48.386288 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b49e04-7a02-44ed-8cfb-9678debfa9e7 None admin] PATH_INFO: `/v3/projects/1d9e82a46b1c4b25939e533c26cb44df` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:48.388640 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b49e04-7a02-44ed-8cfb-9678debfa9e7 None admin] RBAC: Authorizing `identity:get_project(project_id=1d9e82a46b1c4b25939e533c26cb44df)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:48.390317 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b49e04-7a02-44ed-8cfb-9678debfa9e7 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:48.391582 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-33892c16-3cb3-4aa8-8dcb-5c48c243bb6f None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:22:48.392794 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-33892c16-3cb3-4aa8-8dcb-5c48c243bb6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:22:48.392918 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 219/422] 10.222.0.99 () {56 vars in 1186 bytes} [Thu Jul 14 15:22:48 2022] GET /identity/v3/projects/1d9e82a46b1c4b25939e533c26cb44df => generated 298 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:48.400889 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b49e04-7a02-44ed-8cfb-9678debfa9e7 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:48.411976 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:48.490203 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 204/423] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:22:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 15:22:48.493455 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:48.513423 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33892c16-3cb3-4aa8-8dcb-5c48c243bb6f None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:48.583384 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b49e04-7a02-44ed-8cfb-9678debfa9e7 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:48.616802 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33892c16-3cb3-4aa8-8dcb-5c48c243bb6f None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:48.652975 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938baec5-7580-423e-805f-4df8fe256075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:48.653962 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938baec5-7580-423e-805f-4df8fe256075 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:48.654251 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938baec5-7580-423e-805f-4df8fe256075 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:48.654534 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938baec5-7580-423e-805f-4df8fe256075 None admin] PATH_INFO: `/v3/groups/4e08a2fcdc89418db3618b56caf282d4` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:48.656130 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938baec5-7580-423e-805f-4df8fe256075 None admin] RBAC: Authorizing `identity:get_group(group_id=4e08a2fcdc89418db3618b56caf282d4)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:48.656240 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158ce08a-cfe8-4556-b485-1417fa145199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:48.656960 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158ce08a-cfe8-4556-b485-1417fa145199 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:48.657249 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158ce08a-cfe8-4556-b485-1417fa145199 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:48.657523 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158ce08a-cfe8-4556-b485-1417fa145199 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:48.657621 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938baec5-7580-423e-805f-4df8fe256075 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:48.659773 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 220/424] 10.222.0.99 () {56 vars in 1180 bytes} [Thu Jul 14 15:22:48 2022] GET /identity/v3/groups/4e08a2fcdc89418db3618b56caf282d4 => generated 226 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:22:48.665330 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938baec5-7580-423e-805f-4df8fe256075 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:48.669131 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158ce08a-cfe8-4556-b485-1417fa145199 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:22:48.677868 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-158ce08a-cfe8-4556-b485-1417fa145199 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 15:22:48.677868 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:22:48.677868 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:22:48.677868 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:22:48.677868 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:22:48.677868 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:22:48.677868 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:22:48.677868 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:22:48.677868 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:22:48.677868 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:22:48.677868 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:22:48.677868 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:22:48.677868 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:22:48.677868 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:22:48.677868 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:22:48.677868 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:22:48.678785 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:22:48.678785 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:22:48.678785 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:22:48.678785 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:22:48.678785 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:22:48.678785 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:22:48.678785 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:22:48.678785 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:22:48.678785 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:22:48.678785 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:22:48.678785 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:22:48.678785 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:22:48.678785 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:22:48.678785 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:22:48.678785 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:22:48.678785 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:22:48.679655 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:22:48.679655 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:22:48.679655 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:22:48.679655 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:22:48.679655 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:22:48.679655 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:22:48.679655 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:22:48.679655 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:22:48.679655 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:22:48.679655 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:22:48.679655 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 15:22:48.679655 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:22:48.679655 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 205/425] 10.222.0.99 () {56 vars in 1099 bytes} [Thu Jul 14 15:22:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 15:22:48.687179 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158ce08a-cfe8-4556-b485-1417fa145199 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:48.711962 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:48.835687 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938baec5-7580-423e-805f-4df8fe256075 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:48.859427 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:48.886155 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158ce08a-cfe8-4556-b485-1417fa145199 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:48.902274 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb9216b-722d-4a7a-bd4e-a578c33bc277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:48.903539 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb9216b-722d-4a7a-bd4e-a578c33bc277 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:48.903866 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb9216b-722d-4a7a-bd4e-a578c33bc277 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:48.904143 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb9216b-722d-4a7a-bd4e-a578c33bc277 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:48.905172 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb9216b-722d-4a7a-bd4e-a578c33bc277 None admin] RBAC: Adding query filter params (group.id=4e08a2fcdc89418db3618b56caf282d4, role.id=3627b4f45c8c4f41a7def6b15f1b8225, scope.project.id=1d9e82a46b1c4b25939e533c26cb44df) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:22:48.905519 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb9216b-722d-4a7a-bd4e-a578c33bc277 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:48.906990 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb9216b-722d-4a7a-bd4e-a578c33bc277 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:48.922686 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 221/426] 10.222.0.99 () {56 vars in 1376 bytes} [Thu Jul 14 15:22:48 2022] GET /identity/v3/role_assignments?group.id=4e08a2fcdc89418db3618b56caf282d4&scope.project.id=1d9e82a46b1c4b25939e533c26cb44df&role.id=3627b4f45c8c4f41a7def6b15f1b8225 => generated 611 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:22:48.950740 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d77ca6-4c56-4ca6-81ba-7a3a25dca958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:48.951878 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d77ca6-4c56-4ca6-81ba-7a3a25dca958 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:48.952376 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d77ca6-4c56-4ca6-81ba-7a3a25dca958 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:48.952912 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d77ca6-4c56-4ca6-81ba-7a3a25dca958 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:48.953977 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d77ca6-4c56-4ca6-81ba-7a3a25dca958 None admin] RBAC: Adding query filter params (name=member) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:22:48.954527 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d77ca6-4c56-4ca6-81ba-7a3a25dca958 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:48.960572 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d77ca6-4c56-4ca6-81ba-7a3a25dca958 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:48.971954 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 206/427] 10.222.0.99 () {56 vars in 1101 bytes} [Thu Jul 14 15:22:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:48.977944 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb9216b-722d-4a7a-bd4e-a578c33bc277 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:49.038182 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592796db-96c6-4bb1-a940-3dee1280c051 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:49.038814 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592796db-96c6-4bb1-a940-3dee1280c051 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:49.039433 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592796db-96c6-4bb1-a940-3dee1280c051 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:49.041251 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 207/428] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:22:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 15:22:49.053761 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066a1b28-3d9b-44fa-afa0-ad58388f7b19 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:49.054574 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066a1b28-3d9b-44fa-afa0-ad58388f7b19 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:49.055065 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066a1b28-3d9b-44fa-afa0-ad58388f7b19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:49.103725 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-066a1b28-3d9b-44fa-afa0-ad58388f7b19 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:22:49.104860 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-066a1b28-3d9b-44fa-afa0-ad58388f7b19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:22:49.125494 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb9216b-722d-4a7a-bd4e-a578c33bc277 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:49.181506 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92f71af-c475-4e64-b78a-e6e625576831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:49.182798 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92f71af-c475-4e64-b78a-e6e625576831 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:49.183503 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92f71af-c475-4e64-b78a-e6e625576831 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:49.184467 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92f71af-c475-4e64-b78a-e6e625576831 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:49.210131 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 208/429] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:22:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 15:22:49.243773 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:49.251574 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066a1b28-3d9b-44fa-afa0-ad58388f7b19 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:49.257014 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:49.273598 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92f71af-c475-4e64-b78a-e6e625576831 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:49.275356 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92f71af-c475-4e64-b78a-e6e625576831 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:49.393887 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066a1b28-3d9b-44fa-afa0-ad58388f7b19 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:49.406754 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 222/430] 10.222.0.99 () {58 vars in 1282 bytes} [Thu Jul 14 15:22:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 15:22:49.412919 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92f71af-c475-4e64-b78a-e6e625576831 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:49.432137 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83b6f6a-076b-4d5a-969f-092e9a955551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:49.432828 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83b6f6a-076b-4d5a-969f-092e9a955551 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:49.433116 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83b6f6a-076b-4d5a-969f-092e9a955551 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:49.433392 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83b6f6a-076b-4d5a-969f-092e9a955551 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:49.444655 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83b6f6a-076b-4d5a-969f-092e9a955551 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:22:49.453173 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-b83b6f6a-076b-4d5a-969f-092e9a955551 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 15:22:49.453173 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:22:49.453173 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:22:49.453173 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:22:49.453173 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:22:49.453173 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:22:49.453173 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:22:49.453173 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:22:49.453173 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:22:49.453173 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:22:49.453173 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:22:49.453173 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:22:49.453173 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:22:49.453173 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:22:49.453173 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:22:49.453173 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:22:49.454088 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:22:49.454088 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:22:49.454088 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:22:49.454088 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:22:49.454088 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:22:49.454088 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:22:49.454088 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:22:49.454088 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:22:49.454088 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:22:49.454088 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:22:49.454088 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:22:49.454088 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:22:49.454088 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:22:49.454088 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:22:49.454088 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:22:49.454088 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:22:49.454849 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:22:49.454849 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:22:49.454849 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:22:49.454849 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:22:49.454849 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:22:49.454849 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:22:49.454849 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:22:49.454849 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:22:49.454849 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:22:49.454849 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:22:49.454849 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 15:22:49.454849 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:22:49.454849 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 209/431] 10.222.0.99 () {56 vars in 1099 bytes} [Thu Jul 14 15:22:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:22:49.462273 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83b6f6a-076b-4d5a-969f-092e9a955551 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:49.511513 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:49.534487 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:49.536148 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92f71af-c475-4e64-b78a-e6e625576831 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:49.574033 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f351bfb-a723-40de-806b-34b482ee6465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:49.575872 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f351bfb-a723-40de-806b-34b482ee6465 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:49.576454 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f351bfb-a723-40de-806b-34b482ee6465 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:49.576918 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f351bfb-a723-40de-806b-34b482ee6465 None admin] PATH_INFO: `/v3/projects/1d9e82a46b1c4b25939e533c26cb44df` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:49.578361 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f351bfb-a723-40de-806b-34b482ee6465 None admin] RBAC: Authorizing `identity:get_project(project_id=1d9e82a46b1c4b25939e533c26cb44df)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:49.578467 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83b6f6a-076b-4d5a-969f-092e9a955551 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:49.580268 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f351bfb-a723-40de-806b-34b482ee6465 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:49.587006 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 223/432] 10.222.0.99 () {56 vars in 1186 bytes} [Thu Jul 14 15:22:49 2022] GET /identity/v3/projects/1d9e82a46b1c4b25939e533c26cb44df => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:22:49.588126 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f351bfb-a723-40de-806b-34b482ee6465 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:49.616939 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b89af7-9848-4260-acfd-195192e189ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:49.617764 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b89af7-9848-4260-acfd-195192e189ae None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:49.618105 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b89af7-9848-4260-acfd-195192e189ae None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:49.618457 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b89af7-9848-4260-acfd-195192e189ae None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:49.619282 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b89af7-9848-4260-acfd-195192e189ae None admin] RBAC: Adding query filter params (name=member) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:22:49.619692 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b89af7-9848-4260-acfd-195192e189ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:49.621370 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b89af7-9848-4260-acfd-195192e189ae None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:49.631632 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 210/433] 10.222.0.99 () {56 vars in 1101 bytes} [Thu Jul 14 15:22:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:49.640269 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b89af7-9848-4260-acfd-195192e189ae None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:49.706313 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f351bfb-a723-40de-806b-34b482ee6465 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:49.744047 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c417fe4-8b8f-41cc-886b-374a3cc55899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:49.744903 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c417fe4-8b8f-41cc-886b-374a3cc55899 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:49.745184 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c417fe4-8b8f-41cc-886b-374a3cc55899 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:49.745460 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c417fe4-8b8f-41cc-886b-374a3cc55899 None admin] PATH_INFO: `/v3/groups/4e08a2fcdc89418db3618b56caf282d4` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:49.747185 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c417fe4-8b8f-41cc-886b-374a3cc55899 None admin] RBAC: Authorizing `identity:get_group(group_id=4e08a2fcdc89418db3618b56caf282d4)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:49.749059 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c417fe4-8b8f-41cc-886b-374a3cc55899 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:49.750700 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 224/434] 10.222.0.99 () {56 vars in 1180 bytes} [Thu Jul 14 15:22:49 2022] GET /identity/v3/groups/4e08a2fcdc89418db3618b56caf282d4 => generated 226 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:49.751919 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4c8866-f597-4dfe-a6a3-76a8e4d7e61b None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:49.752205 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4c8866-f597-4dfe-a6a3-76a8e4d7e61b None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:49.752482 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4c8866-f597-4dfe-a6a3-76a8e4d7e61b None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:49.753851 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 225/435] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:22:49 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:22:49.759531 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4c8866-f597-4dfe-a6a3-76a8e4d7e61b None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:49.768837 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b89af7-9848-4260-acfd-195192e189ae None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:49.779780 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:49.795826 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:49.805446 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d546a5-39bc-44fa-8a69-5858bad19576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:49.806092 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d546a5-39bc-44fa-8a69-5858bad19576 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:49.806388 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d546a5-39bc-44fa-8a69-5858bad19576 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:49.806664 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d546a5-39bc-44fa-8a69-5858bad19576 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:49.872766 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4c8866-f597-4dfe-a6a3-76a8e4d7e61b None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:49.873916 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d546a5-39bc-44fa-8a69-5858bad19576 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:49.875371 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d546a5-39bc-44fa-8a69-5858bad19576 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:49.911536 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43ac20b-f251-40bb-93fb-5b2d67a1ca3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:49.912593 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43ac20b-f251-40bb-93fb-5b2d67a1ca3c None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:49.913087 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43ac20b-f251-40bb-93fb-5b2d67a1ca3c None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:49.913543 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43ac20b-f251-40bb-93fb-5b2d67a1ca3c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:49.914566 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43ac20b-f251-40bb-93fb-5b2d67a1ca3c None admin] RBAC: Adding query filter params (group.id=4e08a2fcdc89418db3618b56caf282d4, role.id=b8c98fddc0fd4436b577fd1d3361bc38, scope.project.id=1d9e82a46b1c4b25939e533c26cb44df) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:22:49.915059 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43ac20b-f251-40bb-93fb-5b2d67a1ca3c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:49.916869 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43ac20b-f251-40bb-93fb-5b2d67a1ca3c None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:49.931833 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 226/436] 10.222.0.99 () {56 vars in 1376 bytes} [Thu Jul 14 15:22:49 2022] GET /identity/v3/role_assignments?group.id=4e08a2fcdc89418db3618b56caf282d4&scope.project.id=1d9e82a46b1c4b25939e533c26cb44df&role.id=b8c98fddc0fd4436b577fd1d3361bc38 => generated 611 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:22:49.935639 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e816d42-8a7d-4930-a794-1d4de3d64012 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:49.935639 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e816d42-8a7d-4930-a794-1d4de3d64012 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:49.935639 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e816d42-8a7d-4930-a794-1d4de3d64012 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:49.956233 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-7e816d42-8a7d-4930-a794-1d4de3d64012 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:22:49.957553 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-7e816d42-8a7d-4930-a794-1d4de3d64012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:22:50.004694 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 211/437] 10.222.0.99 () {58 vars in 1282 bytes} [Thu Jul 14 15:22:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 14 15:22:50.010702 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d546a5-39bc-44fa-8a69-5858bad19576 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:50.048602 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:50.057265 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 227/438] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:22:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:22:50.058520 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06e939f-57bc-4a2b-86ff-3b3a7ed9e520 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:50.058816 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06e939f-57bc-4a2b-86ff-3b3a7ed9e520 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:50.059183 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06e939f-57bc-4a2b-86ff-3b3a7ed9e520 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:50.071630 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 228/439] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:22:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:22:50.071630 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060ed26c-607e-46f6-a9eb-6142e4e03f1d None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:50.071630 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060ed26c-607e-46f6-a9eb-6142e4e03f1d None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:50.071630 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060ed26c-607e-46f6-a9eb-6142e4e03f1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:50.086245 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-060ed26c-607e-46f6-a9eb-6142e4e03f1d None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:22:50.087493 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-060ed26c-607e-46f6-a9eb-6142e4e03f1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:22:50.092971 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:50.154292 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d546a5-39bc-44fa-8a69-5858bad19576 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:50.172426 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 229/440] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:22:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:22:50.176180 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060ed26c-607e-46f6-a9eb-6142e4e03f1d None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:50.193175 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580da4a7-c834-40af-b4a6-9480cf714e29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:50.194044 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580da4a7-c834-40af-b4a6-9480cf714e29 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:50.194331 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580da4a7-c834-40af-b4a6-9480cf714e29 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:50.194612 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580da4a7-c834-40af-b4a6-9480cf714e29 None admin] PATH_INFO: `/v3/projects/925bf9baf40d4a888a9d633d1deb5134` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:50.198307 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580da4a7-c834-40af-b4a6-9480cf714e29 None admin] RBAC: Authorizing `identity:get_project(project_id=925bf9baf40d4a888a9d633d1deb5134)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:50.200484 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580da4a7-c834-40af-b4a6-9480cf714e29 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:50.202563 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 212/441] 10.222.0.99 () {56 vars in 1186 bytes} [Thu Jul 14 15:22:50 2022] GET /identity/v3/projects/925bf9baf40d4a888a9d633d1deb5134 => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:50.203405 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580da4a7-c834-40af-b4a6-9480cf714e29 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:50.303625 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060ed26c-607e-46f6-a9eb-6142e4e03f1d None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:50.316041 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:50.327806 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580da4a7-c834-40af-b4a6-9480cf714e29 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:50.355763 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c377dc-f361-4e1a-9319-41e44a71a0b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:50.357948 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c377dc-f361-4e1a-9319-41e44a71a0b8 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:50.357948 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c377dc-f361-4e1a-9319-41e44a71a0b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:50.357948 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c377dc-f361-4e1a-9319-41e44a71a0b8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:50.366735 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:50.369646 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22b7cb8-f7ff-4e89-83f4-70b5a8619fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:50.370729 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22b7cb8-f7ff-4e89-83f4-70b5a8619fcb None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:50.371269 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22b7cb8-f7ff-4e89-83f4-70b5a8619fcb None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:50.372504 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22b7cb8-f7ff-4e89-83f4-70b5a8619fcb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:50.372629 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c377dc-f361-4e1a-9319-41e44a71a0b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:22:50.382509 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-d3c377dc-f361-4e1a-9319-41e44a71a0b8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 15:22:50.382509 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:22:50.382509 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:22:50.382509 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:22:50.382509 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:22:50.382509 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:22:50.382509 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:22:50.382509 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:22:50.382509 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:22:50.382509 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:22:50.382509 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:22:50.382509 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:22:50.382509 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:22:50.382509 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:22:50.382509 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:22:50.382509 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:22:50.383515 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:22:50.383515 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:22:50.383515 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:22:50.383515 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:22:50.383515 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:22:50.383515 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:22:50.383515 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:22:50.383515 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:22:50.383515 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:22:50.383515 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:22:50.383515 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:22:50.383515 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:22:50.383515 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:22:50.383515 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:22:50.383515 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:22:50.383515 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:22:50.384339 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:22:50.384339 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:22:50.384339 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:22:50.384339 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:22:50.384339 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:22:50.384339 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:22:50.384339 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:22:50.384339 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:22:50.384339 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:22:50.384339 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:22:50.384339 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 15:22:50.384339 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:22:50.384339 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 230/442] 10.222.0.99 () {56 vars in 1114 bytes} [Thu Jul 14 15:22:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:22:50.384989 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c377dc-f361-4e1a-9319-41e44a71a0b8 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:50.390668 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22b7cb8-f7ff-4e89-83f4-70b5a8619fcb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:22:50.398489 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-a22b7cb8-f7ff-4e89-83f4-70b5a8619fcb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:22:50.398489 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:22:50.398489 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:22:50.398489 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:22:50.398489 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:22:50.398489 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:22:50.398489 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:22:50.398489 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:22:50.398489 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:22:50.398489 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:22:50.398489 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:22:50.398489 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:22:50.398489 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:22:50.398489 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:22:50.398489 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:22:50.398489 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:22:50.402518 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:22:50.402518 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:22:50.402518 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:22:50.402518 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:22:50.402518 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:22:50.402518 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:22:50.402518 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:22:50.402518 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:22:50.402518 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:22:50.402518 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:22:50.402518 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:22:50.402518 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:22:50.402518 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:22:50.402518 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:22:50.402518 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:22:50.402518 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:22:50.404523 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:22:50.404523 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:22:50.404523 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:22:50.404523 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:22:50.404523 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:22:50.404523 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:22:50.404523 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:22:50.404523 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:22:50.404523 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:22:50.404523 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:22:50.404523 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:22:50.404523 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:22:50.404523 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 213/443] 10.222.0.99 () {56 vars in 1096 bytes} [Thu Jul 14 15:22:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:22:50.404523 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22b7cb8-f7ff-4e89-83f4-70b5a8619fcb None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:50.534908 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c377dc-f361-4e1a-9319-41e44a71a0b8 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:50.535419 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22b7cb8-f7ff-4e89-83f4-70b5a8619fcb None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:50.570413 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2382e77-a973-423b-9db7-5c7f1c851b07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:50.571083 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5b6288-3379-4427-9c0f-7e9258fb19dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:50.571215 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2382e77-a973-423b-9db7-5c7f1c851b07 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:50.571477 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2382e77-a973-423b-9db7-5c7f1c851b07 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:50.571791 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2382e77-a973-423b-9db7-5c7f1c851b07 None admin] PATH_INFO: `/v3/groups/4e08a2fcdc89418db3618b56caf282d4` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:50.571898 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5b6288-3379-4427-9c0f-7e9258fb19dc None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:50.572085 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5b6288-3379-4427-9c0f-7e9258fb19dc None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:50.572369 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5b6288-3379-4427-9c0f-7e9258fb19dc None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:50.573024 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2382e77-a973-423b-9db7-5c7f1c851b07 None admin] RBAC: Authorizing `identity:get_group(group_id=4e08a2fcdc89418db3618b56caf282d4)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:50.573117 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5b6288-3379-4427-9c0f-7e9258fb19dc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:22:50.573434 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5b6288-3379-4427-9c0f-7e9258fb19dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:50.574486 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2382e77-a973-423b-9db7-5c7f1c851b07 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:50.574905 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5b6288-3379-4427-9c0f-7e9258fb19dc None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:50.576368 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 231/444] 10.222.0.99 () {56 vars in 1180 bytes} [Thu Jul 14 15:22:50 2022] GET /identity/v3/groups/4e08a2fcdc89418db3618b56caf282d4 => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:50.577161 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2382e77-a973-423b-9db7-5c7f1c851b07 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:50.585329 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 214/445] 10.222.0.99 () {56 vars in 1111 bytes} [Thu Jul 14 15:22:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:50.586227 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5b6288-3379-4427-9c0f-7e9258fb19dc None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:50.609620 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:50.652283 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:50.699521 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2382e77-a973-423b-9db7-5c7f1c851b07 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:50.712016 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5b6288-3379-4427-9c0f-7e9258fb19dc None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:50.736307 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584b3991-1892-460e-9334-e1a23484a758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:50.736988 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584b3991-1892-460e-9334-e1a23484a758 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:50.737278 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584b3991-1892-460e-9334-e1a23484a758 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:50.737561 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584b3991-1892-460e-9334-e1a23484a758 None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:50.738316 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584b3991-1892-460e-9334-e1a23484a758 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:22:50.738626 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584b3991-1892-460e-9334-e1a23484a758 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:50.740050 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584b3991-1892-460e-9334-e1a23484a758 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:50.749125 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b39811-347d-4d97-9e65-351de9b82a1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:50.749575 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 232/446] 10.222.0.99 () {56 vars in 1099 bytes} [Thu Jul 14 15:22:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:50.750004 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b39811-347d-4d97-9e65-351de9b82a1a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:50.750398 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584b3991-1892-460e-9334-e1a23484a758 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:50.750557 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b39811-347d-4d97-9e65-351de9b82a1a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:50.750918 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b39811-347d-4d97-9e65-351de9b82a1a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:50.751955 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b39811-347d-4d97-9e65-351de9b82a1a None admin] RBAC: Adding query filter params (group.id=4e08a2fcdc89418db3618b56caf282d4, role.id=b8c98fddc0fd4436b577fd1d3361bc38, scope.project.id=925bf9baf40d4a888a9d633d1deb5134) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:22:50.752410 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b39811-347d-4d97-9e65-351de9b82a1a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:50.754336 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b39811-347d-4d97-9e65-351de9b82a1a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:50.760836 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 215/447] 10.222.0.99 () {56 vars in 1376 bytes} [Thu Jul 14 15:22:50 2022] GET /identity/v3/role_assignments?group.id=4e08a2fcdc89418db3618b56caf282d4&scope.project.id=925bf9baf40d4a888a9d633d1deb5134&role.id=b8c98fddc0fd4436b577fd1d3361bc38 => generated 611 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:50.761662 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b39811-347d-4d97-9e65-351de9b82a1a None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:50.883303 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584b3991-1892-460e-9334-e1a23484a758 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:50.890038 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:50.914453 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b39811-347d-4d97-9e65-351de9b82a1a None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:50.921235 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:50.930230 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fcf31d-84ee-41d7-93df-0c843e8d0539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:50.930888 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fcf31d-84ee-41d7-93df-0c843e8d0539 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:50.931179 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fcf31d-84ee-41d7-93df-0c843e8d0539 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:50.931467 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fcf31d-84ee-41d7-93df-0c843e8d0539 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:50.971298 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e1e02e-95b7-4fa8-893f-f5307ff29cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:50.972744 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e1e02e-95b7-4fa8-893f-f5307ff29cf6 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:50.973448 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e1e02e-95b7-4fa8-893f-f5307ff29cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:50.974141 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e1e02e-95b7-4fa8-893f-f5307ff29cf6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:51.040402 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fcf31d-84ee-41d7-93df-0c843e8d0539 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:51.041898 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fcf31d-84ee-41d7-93df-0c843e8d0539 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:51.098722 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e1e02e-95b7-4fa8-893f-f5307ff29cf6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:51.102589 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e1e02e-95b7-4fa8-893f-f5307ff29cf6 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:51.193235 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 233/448] 10.222.0.99 () {58 vars in 1282 bytes} [Thu Jul 14 15:22:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 15:22:51.202554 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fcf31d-84ee-41d7-93df-0c843e8d0539 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:51.212910 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:51.235704 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 216/449] 10.222.0.99 () {58 vars in 1282 bytes} [Thu Jul 14 15:22:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 15:22:51.235897 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:51.244479 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e1e02e-95b7-4fa8-893f-f5307ff29cf6 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:51.310793 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fcf31d-84ee-41d7-93df-0c843e8d0539 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:51.352023 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d31f4aa-10d6-4d2f-9247-da2b2e6c4c10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:51.352987 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d31f4aa-10d6-4d2f-9247-da2b2e6c4c10 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:51.353405 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d31f4aa-10d6-4d2f-9247-da2b2e6c4c10 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:51.353766 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d31f4aa-10d6-4d2f-9247-da2b2e6c4c10 None admin] PATH_INFO: `/v3/projects/925bf9baf40d4a888a9d633d1deb5134` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:51.355199 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e1e02e-95b7-4fa8-893f-f5307ff29cf6 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:51.355348 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d31f4aa-10d6-4d2f-9247-da2b2e6c4c10 None admin] RBAC: Authorizing `identity:get_project(project_id=925bf9baf40d4a888a9d633d1deb5134)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:51.356947 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d31f4aa-10d6-4d2f-9247-da2b2e6c4c10 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:51.359111 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 234/450] 10.222.0.99 () {56 vars in 1186 bytes} [Thu Jul 14 15:22:51 2022] GET /identity/v3/projects/925bf9baf40d4a888a9d633d1deb5134 => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:22:51.364555 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d31f4aa-10d6-4d2f-9247-da2b2e6c4c10 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:51.398454 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8697784-3b7f-407b-a025-ee054a313303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:51.399319 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8697784-3b7f-407b-a025-ee054a313303 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:51.399619 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8697784-3b7f-407b-a025-ee054a313303 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:51.399921 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8697784-3b7f-407b-a025-ee054a313303 None admin] PATH_INFO: `/v3/projects/ae21968b232140b899e10f7d7d033a88` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:51.401123 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8697784-3b7f-407b-a025-ee054a313303 None admin] RBAC: Authorizing `identity:get_project(project_id=ae21968b232140b899e10f7d7d033a88)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:51.402583 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8697784-3b7f-407b-a025-ee054a313303 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:51.404583 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 217/451] 10.222.0.99 () {56 vars in 1186 bytes} [Thu Jul 14 15:22:51 2022] GET /identity/v3/projects/ae21968b232140b899e10f7d7d033a88 => generated 344 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:51.410239 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8697784-3b7f-407b-a025-ee054a313303 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:51.460328 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:51.467653 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:51.471986 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d31f4aa-10d6-4d2f-9247-da2b2e6c4c10 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:51.505722 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4232922e-fa90-4e75-94f1-1c2a8a9dae36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:51.506844 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4232922e-fa90-4e75-94f1-1c2a8a9dae36 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:51.507123 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4232922e-fa90-4e75-94f1-1c2a8a9dae36 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:51.507395 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4232922e-fa90-4e75-94f1-1c2a8a9dae36 None admin] PATH_INFO: `/v3/groups/4e08a2fcdc89418db3618b56caf282d4` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:51.508607 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4232922e-fa90-4e75-94f1-1c2a8a9dae36 None admin] RBAC: Authorizing `identity:get_group(group_id=4e08a2fcdc89418db3618b56caf282d4)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:51.510044 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4232922e-fa90-4e75-94f1-1c2a8a9dae36 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:51.511256 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8697784-3b7f-407b-a025-ee054a313303 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:51.511674 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 235/452] 10.222.0.99 () {56 vars in 1180 bytes} [Thu Jul 14 15:22:51 2022] GET /identity/v3/groups/4e08a2fcdc89418db3618b56caf282d4 => generated 226 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:22:51.519127 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4232922e-fa90-4e75-94f1-1c2a8a9dae36 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:51.546555 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9616e5c-7ea1-4423-877e-dcfbad22dce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:51.547416 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9616e5c-7ea1-4423-877e-dcfbad22dce9 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:51.547733 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9616e5c-7ea1-4423-877e-dcfbad22dce9 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:51.548017 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9616e5c-7ea1-4423-877e-dcfbad22dce9 None admin] PATH_INFO: `/v3/groups/9d5e500d21354956bcd54353385f28f2` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:51.549208 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9616e5c-7ea1-4423-877e-dcfbad22dce9 None admin] RBAC: Authorizing `identity:get_group(group_id=9d5e500d21354956bcd54353385f28f2)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:51.550666 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9616e5c-7ea1-4423-877e-dcfbad22dce9 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:51.552326 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 218/453] 10.222.0.99 () {56 vars in 1180 bytes} [Thu Jul 14 15:22:51 2022] GET /identity/v3/groups/9d5e500d21354956bcd54353385f28f2 => generated 229 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:22:51.558085 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9616e5c-7ea1-4423-877e-dcfbad22dce9 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:51.622423 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4232922e-fa90-4e75-94f1-1c2a8a9dae36 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:51.661013 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9616e5c-7ea1-4423-877e-dcfbad22dce9 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:51.661221 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb016fd-9cdd-4e19-8d29-80a1bc5f89eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:51.661823 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb016fd-9cdd-4e19-8d29-80a1bc5f89eb None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:51.662108 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb016fd-9cdd-4e19-8d29-80a1bc5f89eb None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:51.662377 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb016fd-9cdd-4e19-8d29-80a1bc5f89eb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:51.663228 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb016fd-9cdd-4e19-8d29-80a1bc5f89eb None admin] RBAC: Adding query filter params (group.id=4e08a2fcdc89418db3618b56caf282d4, role.id=3627b4f45c8c4f41a7def6b15f1b8225, scope.project.id=925bf9baf40d4a888a9d633d1deb5134) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:22:51.663547 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb016fd-9cdd-4e19-8d29-80a1bc5f89eb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:51.665063 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb016fd-9cdd-4e19-8d29-80a1bc5f89eb None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:51.672157 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 236/454] 10.222.0.99 () {56 vars in 1376 bytes} [Thu Jul 14 15:22:51 2022] GET /identity/v3/role_assignments?group.id=4e08a2fcdc89418db3618b56caf282d4&scope.project.id=925bf9baf40d4a888a9d633d1deb5134&role.id=3627b4f45c8c4f41a7def6b15f1b8225 => generated 611 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:22:51.688317 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:51.709259 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9d2286-a72f-40aa-a521-acd51e7c4bb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:51.709930 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9d2286-a72f-40aa-a521-acd51e7c4bb7 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:51.710213 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9d2286-a72f-40aa-a521-acd51e7c4bb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:51.710543 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9d2286-a72f-40aa-a521-acd51e7c4bb7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:51.711336 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9d2286-a72f-40aa-a521-acd51e7c4bb7 None admin] RBAC: Adding query filter params (group.id=9d5e500d21354956bcd54353385f28f2, role.id=d0f6ba1314f14d9f9c816bdb5d0c3aac, scope.project.id=ae21968b232140b899e10f7d7d033a88) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:22:51.711684 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9d2286-a72f-40aa-a521-acd51e7c4bb7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:51.713138 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9d2286-a72f-40aa-a521-acd51e7c4bb7 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:51.720619 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 219/455] 10.222.0.99 () {56 vars in 1376 bytes} [Thu Jul 14 15:22:51 2022] GET /identity/v3/role_assignments?group.id=9d5e500d21354956bcd54353385f28f2&scope.project.id=ae21968b232140b899e10f7d7d033a88&role.id=d0f6ba1314f14d9f9c816bdb5d0c3aac => generated 611 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:22:54.746752 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19120871-31dd-4fe3-861e-2f6b6e38a12c None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:54.747723 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19120871-31dd-4fe3-861e-2f6b6e38a12c None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:54.748156 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19120871-31dd-4fe3-861e-2f6b6e38a12c None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:54.749928 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 237/456] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:22:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:22:54.755837 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb75bb61-2163-485a-93c8-d9cbce9078c2 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:54.756896 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb75bb61-2163-485a-93c8-d9cbce9078c2 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:54.757327 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb75bb61-2163-485a-93c8-d9cbce9078c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:54.780740 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-cb75bb61-2163-485a-93c8-d9cbce9078c2 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:22:54.782178 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-cb75bb61-2163-485a-93c8-d9cbce9078c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:22:54.838563 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7587f35c-e2ea-42e9-85b7-6ee2d1165b5a None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:54.839118 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7587f35c-e2ea-42e9-85b7-6ee2d1165b5a None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:54.839549 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7587f35c-e2ea-42e9-85b7-6ee2d1165b5a None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:54.841435 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 238/457] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:22:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:22:54.846307 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223ac4bf-3bcb-4c84-8592-b85cd1e84b5c None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:54.846844 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223ac4bf-3bcb-4c84-8592-b85cd1e84b5c None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:54.847348 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223ac4bf-3bcb-4c84-8592-b85cd1e84b5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:54.878773 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-223ac4bf-3bcb-4c84-8592-b85cd1e84b5c None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:22:54.880136 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-223ac4bf-3bcb-4c84-8592-b85cd1e84b5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:22:54.891828 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 220/458] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:22:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 15:22:54.935465 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:54.937320 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb75bb61-2163-485a-93c8-d9cbce9078c2 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:55.014911 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 239/459] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:22:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 15:22:55.016328 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a12aad-03e9-4eed-9ee4-d350045c1b87 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:55.016788 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a12aad-03e9-4eed-9ee4-d350045c1b87 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:55.017222 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a12aad-03e9-4eed-9ee4-d350045c1b87 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:55.018666 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 240/460] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:22:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:22:55.024632 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3544a27c-1042-4d13-a821-d4755d474cb5 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:55.025083 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3544a27c-1042-4d13-a821-d4755d474cb5 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:55.025515 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3544a27c-1042-4d13-a821-d4755d474cb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:55.050922 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-3544a27c-1042-4d13-a821-d4755d474cb5 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:22:55.052433 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-3544a27c-1042-4d13-a821-d4755d474cb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:22:55.100055 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb75bb61-2163-485a-93c8-d9cbce9078c2 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:55.112446 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:55.132411 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 241/461] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:22:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:22:55.133486 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3544a27c-1042-4d13-a821-d4755d474cb5 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:55.142472 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ee7922-b98c-4760-a2cd-ad9d7bf32070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:55.144118 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ee7922-b98c-4760-a2cd-ad9d7bf32070 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:55.144825 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ee7922-b98c-4760-a2cd-ad9d7bf32070 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:55.145498 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ee7922-b98c-4760-a2cd-ad9d7bf32070 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:55.162043 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ee7922-b98c-4760-a2cd-ad9d7bf32070 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:55.164456 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ee7922-b98c-4760-a2cd-ad9d7bf32070 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:55.192093 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-65ee7922-b98c-4760-a2cd-ad9d7bf32070 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:22:55.192093 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:22:55.192093 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:22:55.192093 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:22:55.192093 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:22:55.192093 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:22:55.192093 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:22:55.192093 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:22:55.192093 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:22:55.192093 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:22:55.192093 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:22:55.192093 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:22:55.192093 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:22:55.192093 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:22:55.192093 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:22:55.193039 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:22:55.193039 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:22:55.193039 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:22:55.193039 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:22:55.193039 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:22:55.193039 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:22:55.193039 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:22:55.193039 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:22:55.193039 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:22:55.193039 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:22:55.193039 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:22:55.193039 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:22:55.193039 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:22:55.193039 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:22:55.193039 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:22:55.193039 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:22:55.193039 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:22:55.193868 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:22:55.193868 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:22:55.193868 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:22:55.193868 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:22:55.193868 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 221/462] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:22:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:22:55.195127 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39f776a-0cc2-48f6-9299-613eb86dc37c None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:55.195127 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39f776a-0cc2-48f6-9299-613eb86dc37c None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:55.195311 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39f776a-0cc2-48f6-9299-613eb86dc37c None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:55.201320 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 222/463] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:22:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:22:55.201320 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39f776a-0cc2-48f6-9299-613eb86dc37c None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:55.251841 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:55.262810 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3544a27c-1042-4d13-a821-d4755d474cb5 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:55.303967 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f66503-65b6-494b-b76f-5cbbd9a776dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:55.304710 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f66503-65b6-494b-b76f-5cbbd9a776dc None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:55.305001 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f66503-65b6-494b-b76f-5cbbd9a776dc None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:55.305290 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f66503-65b6-494b-b76f-5cbbd9a776dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:55.316071 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f66503-65b6-494b-b76f-5cbbd9a776dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:55.317556 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f66503-65b6-494b-b76f-5cbbd9a776dc None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:55.323770 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39f776a-0cc2-48f6-9299-613eb86dc37c None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:55.336142 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-d8f66503-65b6-494b-b76f-5cbbd9a776dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:22:55.336142 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:22:55.336142 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:22:55.336142 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:22:55.336142 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:22:55.336142 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:22:55.336142 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:22:55.336142 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:22:55.336142 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:22:55.336142 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:22:55.336142 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:22:55.336142 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:22:55.336142 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:22:55.336142 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:22:55.336142 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:22:55.337014 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:22:55.337014 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:22:55.337014 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:22:55.337014 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:22:55.337014 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:22:55.337014 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:22:55.337014 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:22:55.337014 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:22:55.337014 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:22:55.337014 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:22:55.337014 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:22:55.337014 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:22:55.337014 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:22:55.337014 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:22:55.337014 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:22:55.337014 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:22:55.337014 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:22:55.337850 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:22:55.337850 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:22:55.337850 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:22:55.337850 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:22:55.337850 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 242/464] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:22:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:22:55.338121 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f66503-65b6-494b-b76f-5cbbd9a776dc None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:55.370311 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574f0028-a8be-4518-ace7-b12603a80424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:55.371106 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574f0028-a8be-4518-ace7-b12603a80424 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:55.371399 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574f0028-a8be-4518-ace7-b12603a80424 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:55.371707 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574f0028-a8be-4518-ace7-b12603a80424 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:55.382122 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574f0028-a8be-4518-ace7-b12603a80424 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:55.383574 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574f0028-a8be-4518-ace7-b12603a80424 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:55.402471 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-574f0028-a8be-4518-ace7-b12603a80424 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:22:55.402471 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:22:55.402471 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:22:55.402471 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:22:55.402471 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:22:55.402471 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:22:55.402471 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:22:55.402471 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:22:55.402471 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:22:55.402471 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:22:55.402471 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:22:55.402471 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:22:55.402471 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:22:55.402471 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:22:55.402471 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:22:55.403940 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:22:55.403940 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:22:55.403940 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:22:55.403940 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:22:55.403940 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:22:55.403940 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:22:55.403940 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:22:55.403940 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:22:55.403940 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:22:55.403940 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:22:55.403940 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:22:55.403940 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:22:55.403940 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:22:55.403940 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:22:55.403940 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:22:55.403940 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:22:55.403940 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:22:55.404926 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:22:55.404926 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:22:55.404926 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:22:55.404926 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:22:55.409483 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 223/465] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:22:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:22:55.410315 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4678f1-274a-4c91-9b9a-b5b45c77c906 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:55.410605 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4678f1-274a-4c91-9b9a-b5b45c77c906 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:55.410886 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4678f1-274a-4c91-9b9a-b5b45c77c906 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:55.428391 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-5e4678f1-274a-4c91-9b9a-b5b45c77c906 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:22:55.429565 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-5e4678f1-274a-4c91-9b9a-b5b45c77c906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:22:55.439324 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:55.451954 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f66503-65b6-494b-b76f-5cbbd9a776dc None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:55.491916 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265b2241-6759-4cc8-ad53-d656b8fa690a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:55.492593 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265b2241-6759-4cc8-ad53-d656b8fa690a None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:55.492885 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265b2241-6759-4cc8-ad53-d656b8fa690a None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:55.493158 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265b2241-6759-4cc8-ad53-d656b8fa690a None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:55.493889 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265b2241-6759-4cc8-ad53-d656b8fa690a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:22:55.494197 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265b2241-6759-4cc8-ad53-d656b8fa690a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:55.495575 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265b2241-6759-4cc8-ad53-d656b8fa690a None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:55.507719 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 224/466] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:22:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:22:55.507719 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4678f1-274a-4c91-9b9a-b5b45c77c906 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:55.508325 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 243/467] 10.222.0.99 () {56 vars in 1109 bytes} [Thu Jul 14 15:22:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:55.513941 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265b2241-6759-4cc8-ad53-d656b8fa690a None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:55.517268 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:55.626011 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4678f1-274a-4c91-9b9a-b5b45c77c906 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:55.626209 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265b2241-6759-4cc8-ad53-d656b8fa690a None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:55.664857 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f162efe-6283-4d41-b188-e5c3ca73ce1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:55.665101 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602e6fc1-8818-49aa-9475-01f41e8bdaad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:55.665529 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f162efe-6283-4d41-b188-e5c3ca73ce1b None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:55.665632 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602e6fc1-8818-49aa-9475-01f41e8bdaad None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:55.665813 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f162efe-6283-4d41-b188-e5c3ca73ce1b None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:55.665907 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602e6fc1-8818-49aa-9475-01f41e8bdaad None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:55.666088 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f162efe-6283-4d41-b188-e5c3ca73ce1b None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:55.666193 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602e6fc1-8818-49aa-9475-01f41e8bdaad None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:55.666813 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f162efe-6283-4d41-b188-e5c3ca73ce1b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:22:55.666907 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602e6fc1-8818-49aa-9475-01f41e8bdaad None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:22:55.667121 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f162efe-6283-4d41-b188-e5c3ca73ce1b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:55.667212 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602e6fc1-8818-49aa-9475-01f41e8bdaad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:55.668533 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f162efe-6283-4d41-b188-e5c3ca73ce1b None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:55.668628 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602e6fc1-8818-49aa-9475-01f41e8bdaad None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:55.685987 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 225/468] 10.222.0.99 () {56 vars in 1109 bytes} [Thu Jul 14 15:22:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:55.686179 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 244/469] 10.222.0.99 () {56 vars in 1109 bytes} [Thu Jul 14 15:22:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:55.686179 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602e6fc1-8818-49aa-9475-01f41e8bdaad None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:55.686827 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f162efe-6283-4d41-b188-e5c3ca73ce1b None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:55.692021 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:55.754580 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:55.799326 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f162efe-6283-4d41-b188-e5c3ca73ce1b None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:55.799894 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602e6fc1-8818-49aa-9475-01f41e8bdaad None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:55.837027 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be0c93c-15db-4954-b72f-d7af2e8fc3fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:55.837590 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be0c93c-15db-4954-b72f-d7af2e8fc3fd None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:55.837879 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be0c93c-15db-4954-b72f-d7af2e8fc3fd None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:55.838154 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be0c93c-15db-4954-b72f-d7af2e8fc3fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:55.843701 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9557b9cd-a07c-4523-9186-34ea2d24d65c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:55.844386 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9557b9cd-a07c-4523-9186-34ea2d24d65c None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:55.844675 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9557b9cd-a07c-4523-9186-34ea2d24d65c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:55.844962 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9557b9cd-a07c-4523-9186-34ea2d24d65c None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:55.846115 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9557b9cd-a07c-4523-9186-34ea2d24d65c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:55.848066 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9557b9cd-a07c-4523-9186-34ea2d24d65c None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:55.849524 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be0c93c-15db-4954-b72f-d7af2e8fc3fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:55.851018 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be0c93c-15db-4954-b72f-d7af2e8fc3fd None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:55.870038 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-2be0c93c-15db-4954-b72f-d7af2e8fc3fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:22:55.870038 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:22:55.870038 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:22:55.870038 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:22:55.870038 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:22:55.870038 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:22:55.870038 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:22:55.870038 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:22:55.870038 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:22:55.870038 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:22:55.870038 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:22:55.870038 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:22:55.870038 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:22:55.870038 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:22:55.870038 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:22:55.870946 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:22:55.870946 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:22:55.870946 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:22:55.870946 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:22:55.870946 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:22:55.870946 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:22:55.870946 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:22:55.870946 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:22:55.870946 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:22:55.870946 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:22:55.870946 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:22:55.870946 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:22:55.870946 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:22:55.870946 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:22:55.870946 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:22:55.870946 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:22:55.870946 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:22:55.871874 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:22:55.871874 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:22:55.871874 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:22:55.871874 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:22:55.877338 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 245/470] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:22:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:22:55.878127 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be0c93c-15db-4954-b72f-d7af2e8fc3fd None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:55.879780 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 226/471] 10.222.0.99 () {60 vars in 1132 bytes} [Thu Jul 14 15:22:55 2022] POST /identity/v3/users => generated 248 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:22:55.881327 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9557b9cd-a07c-4523-9186-34ea2d24d65c None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:55.955512 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:55.998757 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be0c93c-15db-4954-b72f-d7af2e8fc3fd None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:56.010686 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9557b9cd-a07c-4523-9186-34ea2d24d65c None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:56.016647 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:56.040881 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67be1986-a2f1-40be-be8e-56616914edad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:56.041568 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67be1986-a2f1-40be-be8e-56616914edad None admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:56.041856 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67be1986-a2f1-40be-be8e-56616914edad None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:56.042138 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67be1986-a2f1-40be-be8e-56616914edad None admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:56.042847 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67be1986-a2f1-40be-be8e-56616914edad None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:56.045178 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67be1986-a2f1-40be-be8e-56616914edad None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:56.057282 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4267a3-a178-4349-848c-a72d9ae884fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:56.058294 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4267a3-a178-4349-848c-a72d9ae884fa None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:56.058634 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4267a3-a178-4349-848c-a72d9ae884fa None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:56.059042 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4267a3-a178-4349-848c-a72d9ae884fa None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:56.060722 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4267a3-a178-4349-848c-a72d9ae884fa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:56.061770 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4267a3-a178-4349-848c-a72d9ae884fa None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:56.068921 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 246/472] 10.222.0.99 () {60 vars in 1132 bytes} [Thu Jul 14 15:22:55 2022] POST /identity/v3/users => generated 246 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:22:56.069772 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67be1986-a2f1-40be-be8e-56616914edad None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:56.104011 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 227/473] 10.222.0.99 () {60 vars in 1132 bytes} [Thu Jul 14 15:22:55 2022] POST /identity/v3/users => generated 249 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:22:56.105586 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03daa7a3-8050-4894-9740-7041a17954b6 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:56.105880 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03daa7a3-8050-4894-9740-7041a17954b6 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:56.106156 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03daa7a3-8050-4894-9740-7041a17954b6 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:56.107833 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 228/474] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:22:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:22:56.114140 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a35856e-d33a-4093-a73f-fb2fbb95b99b None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:56.114332 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a35856e-d33a-4093-a73f-fb2fbb95b99b None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:56.114530 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a35856e-d33a-4093-a73f-fb2fbb95b99b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:56.137515 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-0a35856e-d33a-4093-a73f-fb2fbb95b99b None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:22:56.139679 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0a35856e-d33a-4093-a73f-fb2fbb95b99b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:22:56.208691 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67be1986-a2f1-40be-be8e-56616914edad None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:56.234836 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 229/475] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:22:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 15:22:56.251155 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c267f2c6-d4bd-4704-8561-fc8505dfbb0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:56.252844 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c267f2c6-d4bd-4704-8561-fc8505dfbb0c None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:56.253600 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c267f2c6-d4bd-4704-8561-fc8505dfbb0c None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:56.254301 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c267f2c6-d4bd-4704-8561-fc8505dfbb0c None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:56.255854 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c267f2c6-d4bd-4704-8561-fc8505dfbb0c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:22:56.256397 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c267f2c6-d4bd-4704-8561-fc8505dfbb0c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:56.258387 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c267f2c6-d4bd-4704-8561-fc8505dfbb0c None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:56.268944 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a35856e-d33a-4093-a73f-fb2fbb95b99b None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:56.274637 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 247/476] 10.222.0.99 () {56 vars in 1109 bytes} [Thu Jul 14 15:22:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:56.281295 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c267f2c6-d4bd-4704-8561-fc8505dfbb0c None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:56.337364 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:56.368362 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:56.442205 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a35856e-d33a-4093-a73f-fb2fbb95b99b None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:56.444739 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c267f2c6-d4bd-4704-8561-fc8505dfbb0c None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:56.486836 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ad0dac-3cf0-4139-901c-2c15eb7300cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:56.487989 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ad0dac-3cf0-4139-901c-2c15eb7300cb None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:56.488461 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ad0dac-3cf0-4139-901c-2c15eb7300cb None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:56.488947 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ad0dac-3cf0-4139-901c-2c15eb7300cb None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:56.490001 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ad0dac-3cf0-4139-901c-2c15eb7300cb None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:56.491654 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ad0dac-3cf0-4139-901c-2c15eb7300cb None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:56.499650 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d46930-855d-4978-aef0-4689ff651178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:56.500467 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d46930-855d-4978-aef0-4689ff651178 None admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:56.500759 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d46930-855d-4978-aef0-4689ff651178 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:56.501044 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d46930-855d-4978-aef0-4689ff651178 None admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:56.501789 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d46930-855d-4978-aef0-4689ff651178 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:56.503289 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d46930-855d-4978-aef0-4689ff651178 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:56.505597 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.sql.core [None req-d5ad0dac-3cf0-4139-901c-2c15eb7300cb None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 14 15:22:56.505597 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 14 15:22:56.505597 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [parameters: {'id': '558812bb50e64747b055e9696f15bbe7', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 14 15:22:56.505597 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80237) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 14 15:22:56.523116 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-d5ad0dac-3cf0-4139-901c-2c15eb7300cb None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 14 15:22:56.523116 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:22:56.523116 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 14 15:22:56.523116 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 14 15:22:56.523116 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 14 15:22:56.523116 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 14 15:22:56.523116 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 14 15:22:56.523116 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application result = self._query(query) Jul 14 15:22:56.523116 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 14 15:22:56.523116 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application conn.query(q) Jul 14 15:22:56.523116 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 14 15:22:56.523116 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 14 15:22:56.523116 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 14 15:22:56.523116 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application result.read() Jul 14 15:22:56.523116 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 14 15:22:56.524146 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 14 15:22:56.524146 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 14 15:22:56.524146 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 14 15:22:56.524146 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 14 15:22:56.524146 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 14 15:22:56.524146 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 14 15:22:56.524146 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 14 15:22:56.524146 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 14 15:22:56.524146 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:22:56.524146 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 14 15:22:56.524146 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:22:56.524146 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:22:56.524146 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 14 15:22:56.524146 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 14 15:22:56.524146 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 14 15:22:56.524146 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return ref.to_dict() Jul 14 15:22:56.524146 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 14 15:22:56.524146 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application next(self.gen) Jul 14 15:22:56.525124 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 14 15:22:56.525124 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application yield resource Jul 14 15:22:56.525124 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 14 15:22:56.525124 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application next(self.gen) Jul 14 15:22:56.525124 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 14 15:22:56.525124 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self.session.rollback() Jul 14 15:22:56.525124 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 14 15:22:56.525124 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self.force_reraise() Jul 14 15:22:56.525124 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 14 15:22:56.525124 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise self.value Jul 14 15:22:56.525124 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 14 15:22:56.525124 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 14 15:22:56.525124 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 14 15:22:56.525124 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application session.commit() Jul 14 15:22:56.525124 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 14 15:22:56.525124 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 14 15:22:56.525124 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 14 15:22:56.526517 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._prepare_impl() Jul 14 15:22:56.526517 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 14 15:22:56.526517 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self.session.flush() Jul 14 15:22:56.526517 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 14 15:22:56.526517 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._flush(objects) Jul 14 15:22:56.526517 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 14 15:22:56.526517 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 14 15:22:56.526517 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 14 15:22:56.526517 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application compat.raise_( Jul 14 15:22:56.526517 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 14 15:22:56.526517 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception Jul 14 15:22:56.526517 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 14 15:22:56.526517 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application flush_context.execute() Jul 14 15:22:56.526517 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 14 15:22:56.526517 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rec.execute(self) Jul 14 15:22:56.526517 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 14 15:22:56.526517 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 14 15:22:56.527490 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 14 15:22:56.527490 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 14 15:22:56.527490 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 14 15:22:56.527490 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 14 15:22:56.527490 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 14 15:22:56.527490 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 14 15:22:56.527490 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 14 15:22:56.527490 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 14 15:22:56.527490 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 14 15:22:56.527490 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 14 15:22:56.527490 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 14 15:22:56.527490 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 14 15:22:56.527490 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 14 15:22:56.527490 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 14 15:22:56.527490 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 14 15:22:56.527490 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception Jul 14 15:22:56.528407 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 14 15:22:56.528407 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 14 15:22:56.528407 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 14 15:22:56.528407 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 14 15:22:56.528407 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 14 15:22:56.528407 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application result = self._query(query) Jul 14 15:22:56.528407 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 14 15:22:56.528407 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application conn.query(q) Jul 14 15:22:56.528407 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 14 15:22:56.528407 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 14 15:22:56.528407 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 14 15:22:56.528407 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application result.read() Jul 14 15:22:56.528407 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 14 15:22:56.528407 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 14 15:22:56.528407 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 14 15:22:56.528407 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 14 15:22:56.529215 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 14 15:22:56.529215 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 14 15:22:56.529215 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 14 15:22:56.529215 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 14 15:22:56.529215 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 14 15:22:56.529215 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 14 15:22:56.529215 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [parameters: {'id': '558812bb50e64747b055e9696f15bbe7', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 14 15:22:56.529215 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 14 15:22:56.529215 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:22:56.529215 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 15:22:56.529215 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:22:56.529215 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:22:56.529215 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:22:56.529215 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:22:56.529215 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:22:56.529215 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:22:56.531356 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:22:56.531356 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:22:56.531356 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:22:56.531356 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:22:56.531356 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:22:56.531356 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:22:56.531356 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 14 15:22:56.531356 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 14 15:22:56.531356 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:22:56.531356 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:22:56.531356 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 14 15:22:56.531356 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 14 15:22:56.531356 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 14 15:22:56.531356 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 14 15:22:56.531356 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 14 15:22:56.531356 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:22:56.533887 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 230/477] 10.222.0.99 () {60 vars in 1131 bytes} [Thu Jul 14 15:22:56 2022] POST /identity/v3/roles => generated 144 bytes in 257 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 14 15:22:56.533887 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ad0dac-3cf0-4139-901c-2c15eb7300cb None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:56.541047 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 248/478] 10.222.0.99 () {60 vars in 1132 bytes} [Thu Jul 14 15:22:56 2022] POST /identity/v3/users => generated 248 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 15:22:56.649693 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ad0dac-3cf0-4139-901c-2c15eb7300cb None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:56.664267 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:56.689354 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98651411-7a24-4f1f-8c8e-9e65824dd811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:56.690172 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98651411-7a24-4f1f-8c8e-9e65824dd811 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:56.690455 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98651411-7a24-4f1f-8c8e-9e65824dd811 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:56.690731 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98651411-7a24-4f1f-8c8e-9e65824dd811 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:56.703064 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98651411-7a24-4f1f-8c8e-9e65824dd811 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:22:56.711428 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-98651411-7a24-4f1f-8c8e-9e65824dd811 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 15:22:56.711428 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:22:56.711428 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:22:56.711428 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:22:56.711428 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:22:56.711428 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:22:56.711428 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:22:56.711428 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:22:56.711428 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:22:56.711428 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:22:56.711428 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:22:56.711428 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:22:56.711428 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:22:56.711428 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:22:56.711428 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:22:56.711428 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:22:56.712373 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:22:56.712373 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:22:56.712373 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:22:56.712373 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:22:56.712373 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:22:56.712373 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:22:56.712373 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:22:56.712373 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:22:56.712373 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:22:56.712373 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:22:56.712373 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:22:56.712373 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:22:56.712373 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:22:56.712373 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:22:56.712373 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:22:56.712373 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:22:56.713145 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:22:56.713145 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:22:56.713145 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:22:56.713145 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:22:56.713145 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:22:56.713145 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:22:56.713145 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:22:56.713145 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:22:56.713145 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:22:56.713145 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:22:56.713145 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 15:22:56.713145 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:22:56.713145 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 231/479] 10.222.0.99 () {56 vars in 1114 bytes} [Thu Jul 14 15:22:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:22:56.720819 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d46930-855d-4978-aef0-4689ff651178 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:56.842987 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:56.863442 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d46930-855d-4978-aef0-4689ff651178 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:56.900210 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa3bb26-570b-49a5-a4b6-9e0c3884daf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:56.900816 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa3bb26-570b-49a5-a4b6-9e0c3884daf0 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:56.901104 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa3bb26-570b-49a5-a4b6-9e0c3884daf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:56.901387 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa3bb26-570b-49a5-a4b6-9e0c3884daf0 None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:56.902149 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa3bb26-570b-49a5-a4b6-9e0c3884daf0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:22:56.902458 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa3bb26-570b-49a5-a4b6-9e0c3884daf0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:56.904499 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa3bb26-570b-49a5-a4b6-9e0c3884daf0 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:56.913703 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 249/480] 10.222.0.99 () {56 vars in 1111 bytes} [Thu Jul 14 15:22:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:58.323025 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199d2e0f-ecc9-480e-9187-4b4a0cc0d0cf None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:58.323508 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199d2e0f-ecc9-480e-9187-4b4a0cc0d0cf None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:58.323508 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199d2e0f-ecc9-480e-9187-4b4a0cc0d0cf None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:58.325187 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 232/481] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:22:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:22:58.331248 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87c5336-b1ed-4d96-bfd0-8582737eec93 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:58.331748 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87c5336-b1ed-4d96-bfd0-8582737eec93 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:58.331955 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87c5336-b1ed-4d96-bfd0-8582737eec93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:58.357329 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-d87c5336-b1ed-4d96-bfd0-8582737eec93 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:22:58.358848 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-d87c5336-b1ed-4d96-bfd0-8582737eec93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:22:58.444404 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 250/482] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:22:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:22:58.485187 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199d2e0f-ecc9-480e-9187-4b4a0cc0d0cf None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:58.602385 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199d2e0f-ecc9-480e-9187-4b4a0cc0d0cf None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:58.639451 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93db9593-9c88-4cb4-a496-d2bcda3a6dbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:58.640288 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93db9593-9c88-4cb4-a496-d2bcda3a6dbf None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:58.640577 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93db9593-9c88-4cb4-a496-d2bcda3a6dbf None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:58.640860 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93db9593-9c88-4cb4-a496-d2bcda3a6dbf None admin] PATH_INFO: `/v3/roles/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:58.651795 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93db9593-9c88-4cb4-a496-d2bcda3a6dbf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:22:58.660371 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-93db9593-9c88-4cb4-a496-d2bcda3a6dbf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 15:22:58.660371 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:22:58.660371 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:22:58.660371 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:22:58.660371 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:22:58.660371 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:22:58.660371 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:22:58.660371 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:22:58.660371 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:22:58.660371 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:22:58.660371 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:22:58.660371 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:22:58.660371 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:22:58.660371 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:22:58.660371 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:22:58.660371 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:22:58.661615 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:22:58.661615 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:22:58.661615 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:22:58.661615 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:22:58.661615 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:22:58.661615 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:22:58.661615 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:22:58.661615 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:22:58.661615 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:22:58.661615 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:22:58.661615 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:22:58.661615 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:22:58.661615 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:22:58.661615 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:22:58.661615 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:22:58.661615 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:22:58.662528 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:22:58.662528 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:22:58.662528 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:22:58.662528 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:22:58.662528 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:22:58.662528 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:22:58.662528 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:22:58.662528 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:22:58.662528 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:22:58.662528 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:22:58.662528 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 15:22:58.662528 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:22:58.662528 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 233/483] 10.222.0.99 () {56 vars in 1102 bytes} [Thu Jul 14 15:22:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:22:58.667467 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87c5336-b1ed-4d96-bfd0-8582737eec93 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:58.791095 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:58.823906 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87c5336-b1ed-4d96-bfd0-8582737eec93 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:58.871987 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f6dad4-37e6-4c63-bae2-d502743a716e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:58.873202 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f6dad4-37e6-4c63-bae2-d502743a716e None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:58.873502 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f6dad4-37e6-4c63-bae2-d502743a716e None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:58.873782 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f6dad4-37e6-4c63-bae2-d502743a716e None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:58.874548 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f6dad4-37e6-4c63-bae2-d502743a716e None admin] RBAC: Adding query filter params (name=service) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:22:58.874859 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f6dad4-37e6-4c63-bae2-d502743a716e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:58.876282 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f6dad4-37e6-4c63-bae2-d502743a716e None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:58.889090 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 251/484] 10.222.0.99 () {56 vars in 1103 bytes} [Thu Jul 14 15:22:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:58.894151 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93db9593-9c88-4cb4-a496-d2bcda3a6dbf None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:58.938249 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:58.959434 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a04a8a5-8818-4806-99fa-4e4dfc1bacb8 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:58.959878 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a04a8a5-8818-4806-99fa-4e4dfc1bacb8 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:58.960145 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a04a8a5-8818-4806-99fa-4e4dfc1bacb8 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:58.961627 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 252/485] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:22:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 15:22:58.969365 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4f6d4a-1651-4f28-b172-cdb831e3b736 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:58.969710 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4f6d4a-1651-4f28-b172-cdb831e3b736 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:58.969966 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4f6d4a-1651-4f28-b172-cdb831e3b736 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:58.995821 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-4d4f6d4a-1651-4f28-b172-cdb831e3b736 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:22:58.997829 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-4d4f6d4a-1651-4f28-b172-cdb831e3b736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:22:59.052526 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93db9593-9c88-4cb4-a496-d2bcda3a6dbf None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:59.075926 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 253/486] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:22:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 15:22:59.077461 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa9854a-3807-4370-907a-7e44e7fbd93d None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:59.077673 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa9854a-3807-4370-907a-7e44e7fbd93d None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:59.077942 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa9854a-3807-4370-907a-7e44e7fbd93d None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:59.079564 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 254/487] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:22:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:22:59.085026 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34108806-55bf-46db-9f8b-20091a1b85f3 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:59.085319 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34108806-55bf-46db-9f8b-20091a1b85f3 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:59.085604 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34108806-55bf-46db-9f8b-20091a1b85f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:59.097522 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9aa3157-ed76-492c-9b25-3fe2a12e17f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:59.098699 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aa3157-ed76-492c-9b25-3fe2a12e17f2 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:59.099212 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aa3157-ed76-492c-9b25-3fe2a12e17f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:59.099740 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aa3157-ed76-492c-9b25-3fe2a12e17f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:59.111822 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9aa3157-ed76-492c-9b25-3fe2a12e17f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:59.113575 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9aa3157-ed76-492c-9b25-3fe2a12e17f2 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:59.116899 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-34108806-55bf-46db-9f8b-20091a1b85f3 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:22:59.118104 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-34108806-55bf-46db-9f8b-20091a1b85f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:22:59.134000 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-a9aa3157-ed76-492c-9b25-3fe2a12e17f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:22:59.134000 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:22:59.134000 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:22:59.134000 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:22:59.134000 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:22:59.134000 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:22:59.134000 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:22:59.134000 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:22:59.134000 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:22:59.134000 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:22:59.134000 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:22:59.134000 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:22:59.134000 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:22:59.134000 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:22:59.134000 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:22:59.134874 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:22:59.134874 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:22:59.134874 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:22:59.134874 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:22:59.134874 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:22:59.134874 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:22:59.134874 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:22:59.134874 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:22:59.134874 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:22:59.134874 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:22:59.134874 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:22:59.134874 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:22:59.134874 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:22:59.134874 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:22:59.134874 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:22:59.134874 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:22:59.134874 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:22:59.135735 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:22:59.135735 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:22:59.135735 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:22:59.135735 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:22:59.138589 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 234/488] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:22:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:22:59.139620 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9aa3157-ed76-492c-9b25-3fe2a12e17f2 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:59.184277 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:59.190540 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 255/489] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:22:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 15:22:59.195733 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34108806-55bf-46db-9f8b-20091a1b85f3 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:59.260960 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:59.284652 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9aa3157-ed76-492c-9b25-3fe2a12e17f2 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:59.297466 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34108806-55bf-46db-9f8b-20091a1b85f3 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:59.322372 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bceeee9-410f-476c-a394-e304f6bd306e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:59.323580 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bceeee9-410f-476c-a394-e304f6bd306e None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:59.324526 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bceeee9-410f-476c-a394-e304f6bd306e None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:59.325086 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bceeee9-410f-476c-a394-e304f6bd306e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:59.334729 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ce5e09-c30d-4285-bb8f-73b096e49197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:59.335449 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ce5e09-c30d-4285-bb8f-73b096e49197 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:59.335975 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ce5e09-c30d-4285-bb8f-73b096e49197 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:59.336168 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ce5e09-c30d-4285-bb8f-73b096e49197 None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:59.336854 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ce5e09-c30d-4285-bb8f-73b096e49197 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:22:59.337165 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ce5e09-c30d-4285-bb8f-73b096e49197 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:59.338544 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ce5e09-c30d-4285-bb8f-73b096e49197 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:59.339749 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bceeee9-410f-476c-a394-e304f6bd306e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:22:59.349064 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-8bceeee9-410f-476c-a394-e304f6bd306e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 15:22:59.349064 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:22:59.349064 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:22:59.349064 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:22:59.349064 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:22:59.349064 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:22:59.349064 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:22:59.349064 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:22:59.349064 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:22:59.349064 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:22:59.349064 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:22:59.349064 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:22:59.349064 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:22:59.349064 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:22:59.349064 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:22:59.349064 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:22:59.349958 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:22:59.349958 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:22:59.349958 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:22:59.349958 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:22:59.349958 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:22:59.349958 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:22:59.349958 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:22:59.349958 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:22:59.349958 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:22:59.349958 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:22:59.349958 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:22:59.349958 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:22:59.349958 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:22:59.349958 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:22:59.349958 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:22:59.349958 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:22:59.350739 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:22:59.350739 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:22:59.350739 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:22:59.350739 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:22:59.350739 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:22:59.350739 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:22:59.350739 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:22:59.350739 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:22:59.350739 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:22:59.350739 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:22:59.350739 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 15:22:59.350739 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:22:59.351340 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 256/490] 10.222.0.99 () {56 vars in 1109 bytes} [Thu Jul 14 15:22:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:59.352881 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ce5e09-c30d-4285-bb8f-73b096e49197 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:59.354011 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 235/491] 10.222.0.99 () {56 vars in 1102 bytes} [Thu Jul 14 15:22:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:22:59.360067 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bceeee9-410f-476c-a394-e304f6bd306e None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:59.433574 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:59.491989 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bceeee9-410f-476c-a394-e304f6bd306e None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:59.492621 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ce5e09-c30d-4285-bb8f-73b096e49197 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:59.535122 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ff4d03-be55-40e0-8a62-cf7e96edf1ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:59.535978 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ff4d03-be55-40e0-8a62-cf7e96edf1ad None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:59.536295 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ff4d03-be55-40e0-8a62-cf7e96edf1ad None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:59.536543 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ff4d03-be55-40e0-8a62-cf7e96edf1ad None admin] PATH_INFO: `/v3/users/glance` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:59.538293 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187d7168-3113-4cac-9496-1201cf8288bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:59.540035 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187d7168-3113-4cac-9496-1201cf8288bd None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:59.540547 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187d7168-3113-4cac-9496-1201cf8288bd None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:59.541045 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187d7168-3113-4cac-9496-1201cf8288bd None admin] PATH_INFO: `/v3/roles/service` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:59.545956 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ff4d03-be55-40e0-8a62-cf7e96edf1ad None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:59.547981 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ff4d03-be55-40e0-8a62-cf7e96edf1ad None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:59.555289 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187d7168-3113-4cac-9496-1201cf8288bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:22:59.564249 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-187d7168-3113-4cac-9496-1201cf8288bd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 15:22:59.564249 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:22:59.564249 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:22:59.564249 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:22:59.564249 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:22:59.564249 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:22:59.564249 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:22:59.564249 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:22:59.564249 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:22:59.564249 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:22:59.564249 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:22:59.564249 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:22:59.564249 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:22:59.564249 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:22:59.564249 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:22:59.564249 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:22:59.565130 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:22:59.565130 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:22:59.565130 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:22:59.565130 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:22:59.565130 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:22:59.565130 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:22:59.565130 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:22:59.565130 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:22:59.565130 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:22:59.565130 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:22:59.565130 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:22:59.565130 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:22:59.565130 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:22:59.565130 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:22:59.565130 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:22:59.565130 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:22:59.565915 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:22:59.565915 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:22:59.565915 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:22:59.565915 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:22:59.565915 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:22:59.565915 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:22:59.565915 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:22:59.565915 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:22:59.565915 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:22:59.565915 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:22:59.565915 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 15:22:59.565915 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:22:59.565915 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 257/492] 10.222.0.99 () {56 vars in 1102 bytes} [Thu Jul 14 15:22:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:22:59.566578 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187d7168-3113-4cac-9496-1201cf8288bd None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:59.573447 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-16ff4d03-be55-40e0-8a62-cf7e96edf1ad None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 14 15:22:59.573447 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:22:59.573447 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:22:59.573447 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:22:59.573447 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:22:59.573447 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:22:59.573447 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:22:59.573447 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:22:59.573447 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:22:59.573447 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:22:59.573447 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:22:59.573447 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:22:59.573447 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 15:22:59.573447 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 15:22:59.573447 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 15:22:59.573447 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 15:22:59.574378 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:22:59.574378 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:22:59.574378 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 15:22:59.574378 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:22:59.574378 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 15:22:59.574378 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:22:59.574378 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:22:59.574378 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:22:59.574378 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:22:59.574378 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:22:59.574378 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:22:59.574378 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:22:59.574378 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:22:59.574378 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:22:59.574378 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:22:59.574378 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:22:59.574378 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:22:59.574378 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:22:59.575305 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:22:59.575305 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:22:59.575305 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 15:22:59.575305 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 15:22:59.575305 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 15:22:59.575305 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 15:22:59.575305 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 15:22:59.575305 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 15:22:59.575305 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 14 15:22:59.575305 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:22:59.578522 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 236/493] 10.222.0.99 () {56 vars in 1099 bytes} [Thu Jul 14 15:22:59 2022] GET /identity/v3/users/glance => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:22:59.582486 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ff4d03-be55-40e0-8a62-cf7e96edf1ad None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:59.585301 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:59.678670 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187d7168-3113-4cac-9496-1201cf8288bd None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:59.702890 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ff4d03-be55-40e0-8a62-cf7e96edf1ad None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:59.703376 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:59.721630 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2e87b1-e600-470a-83e6-a03c8a90dfee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:59.722572 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2e87b1-e600-470a-83e6-a03c8a90dfee None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:59.722868 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2e87b1-e600-470a-83e6-a03c8a90dfee None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:59.723117 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2e87b1-e600-470a-83e6-a03c8a90dfee None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:59.723925 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2e87b1-e600-470a-83e6-a03c8a90dfee None admin] RBAC: Adding query filter params (name=service) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:22:59.724236 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2e87b1-e600-470a-83e6-a03c8a90dfee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:59.725630 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2e87b1-e600-470a-83e6-a03c8a90dfee None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:59.735319 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 258/494] 10.222.0.99 () {56 vars in 1103 bytes} [Thu Jul 14 15:22:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:59.745228 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ae3fe3-bdc6-4cb8-8ec1-7fcdedce5024 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:59.745228 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ae3fe3-bdc6-4cb8-8ec1-7fcdedce5024 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:59.745228 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ae3fe3-bdc6-4cb8-8ec1-7fcdedce5024 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:59.745228 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 259/495] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:22:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:22:59.745228 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ae3fe3-bdc6-4cb8-8ec1-7fcdedce5024 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:59.754426 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a6f440-3e67-4eb0-8d18-5e9ffa017605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:59.756056 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a6f440-3e67-4eb0-8d18-5e9ffa017605 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:59.756491 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a6f440-3e67-4eb0-8d18-5e9ffa017605 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:59.756923 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a6f440-3e67-4eb0-8d18-5e9ffa017605 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:59.758494 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a6f440-3e67-4eb0-8d18-5e9ffa017605 None admin] RBAC: Adding query filter params (name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:22:59.759285 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a6f440-3e67-4eb0-8d18-5e9ffa017605 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:59.761227 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a6f440-3e67-4eb0-8d18-5e9ffa017605 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:59.771902 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 237/496] 10.222.0.99 () {56 vars in 1103 bytes} [Thu Jul 14 15:22:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:59.773038 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a6f440-3e67-4eb0-8d18-5e9ffa017605 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:59.846587 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:22:59.869871 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ae3fe3-bdc6-4cb8-8ec1-7fcdedce5024 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:59.887807 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a6f440-3e67-4eb0-8d18-5e9ffa017605 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:22:59.905840 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c2fbd8-1a16-4cae-8572-b69cc64c5306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:59.906520 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c2fbd8-1a16-4cae-8572-b69cc64c5306 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:59.906809 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c2fbd8-1a16-4cae-8572-b69cc64c5306 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:59.907089 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c2fbd8-1a16-4cae-8572-b69cc64c5306 None admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:59.907979 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c2fbd8-1a16-4cae-8572-b69cc64c5306 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:22:59.908293 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c2fbd8-1a16-4cae-8572-b69cc64c5306 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:59.909735 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c2fbd8-1a16-4cae-8572-b69cc64c5306 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:59.922306 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a40807-07d7-48f8-9d29-4fdc64627e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:22:59.923095 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a40807-07d7-48f8-9d29-4fdc64627e0a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:59.923380 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a40807-07d7-48f8-9d29-4fdc64627e0a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:59.923687 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a40807-07d7-48f8-9d29-4fdc64627e0a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:59.934310 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a40807-07d7-48f8-9d29-4fdc64627e0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:22:59.935921 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a40807-07d7-48f8-9d29-4fdc64627e0a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:22:59.947147 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 260/497] 10.222.0.99 () {56 vars in 1137 bytes} [Thu Jul 14 15:22:59 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:22:59.948459 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5455cf2-de67-42f6-901b-ab9a4021b55d None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:22:59.948769 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5455cf2-de67-42f6-901b-ab9a4021b55d None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:22:59.949033 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5455cf2-de67-42f6-901b-ab9a4021b55d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:22:59.953929 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-28a40807-07d7-48f8-9d29-4fdc64627e0a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:22:59.953929 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:22:59.953929 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:22:59.953929 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:22:59.953929 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:22:59.953929 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:22:59.953929 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:22:59.953929 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:22:59.953929 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:22:59.953929 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:22:59.953929 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:22:59.953929 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:22:59.953929 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:22:59.953929 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:22:59.953929 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:22:59.954802 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:22:59.954802 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:22:59.954802 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:22:59.954802 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:22:59.954802 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:22:59.954802 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:22:59.954802 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:22:59.954802 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:22:59.954802 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:22:59.954802 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:22:59.954802 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:22:59.954802 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:22:59.954802 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:22:59.954802 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:22:59.954802 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:22:59.954802 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:22:59.954802 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:22:59.955679 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:22:59.955679 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:22:59.955679 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:22:59.955679 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:22:59.955679 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 238/498] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:22:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:22:59.956030 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a40807-07d7-48f8-9d29-4fdc64627e0a None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:22:59.969871 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-c5455cf2-de67-42f6-901b-ab9a4021b55d None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:22:59.971497 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-c5455cf2-de67-42f6-901b-ab9a4021b55d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:23:00.013606 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:00.052053 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 261/499] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:22:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:23:00.053420 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45041d81-408a-4108-b98c-a2a2f32bd76a None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:00.053711 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45041d81-408a-4108-b98c-a2a2f32bd76a None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:00.053996 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45041d81-408a-4108-b98c-a2a2f32bd76a None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:00.055326 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 262/500] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:23:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:23:00.056402 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45041d81-408a-4108-b98c-a2a2f32bd76a None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:00.084409 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a40807-07d7-48f8-9d29-4fdc64627e0a None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:00.100510 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:00.119421 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acc6edc-e4fb-4b5f-929e-dc5c2cdd3532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:00.120247 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acc6edc-e4fb-4b5f-929e-dc5c2cdd3532 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:00.120537 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acc6edc-e4fb-4b5f-929e-dc5c2cdd3532 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:00.120818 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acc6edc-e4fb-4b5f-929e-dc5c2cdd3532 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:00.132438 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acc6edc-e4fb-4b5f-929e-dc5c2cdd3532 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:00.133894 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acc6edc-e4fb-4b5f-929e-dc5c2cdd3532 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:00.151167 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-2acc6edc-e4fb-4b5f-929e-dc5c2cdd3532 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:23:00.151167 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:00.151167 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:00.151167 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:00.151167 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:00.151167 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:00.151167 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:00.151167 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:00.151167 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:00.151167 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:00.151167 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:00.151167 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:00.151167 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:23:00.151167 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:23:00.151167 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:23:00.152158 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:23:00.152158 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:00.152158 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:00.152158 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:00.152158 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:00.152158 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:00.152158 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:00.152158 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:00.152158 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:00.152158 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:00.152158 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:00.152158 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:00.152158 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:00.152158 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:00.152158 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:00.152158 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:00.152158 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:00.153008 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:23:00.153008 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:23:00.153008 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:23:00.153008 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:23:00.153008 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 239/501] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:22:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:00.153262 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acc6edc-e4fb-4b5f-929e-dc5c2cdd3532 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:00.167764 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45041d81-408a-4108-b98c-a2a2f32bd76a None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:00.202016 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb25db1-43cb-4731-bb5c-095293bcf092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:00.202758 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb25db1-43cb-4731-bb5c-095293bcf092 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:00.203049 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb25db1-43cb-4731-bb5c-095293bcf092 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:00.203328 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb25db1-43cb-4731-bb5c-095293bcf092 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:00.242854 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:00.255983 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acc6edc-e4fb-4b5f-929e-dc5c2cdd3532 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:00.270646 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb25db1-43cb-4731-bb5c-095293bcf092 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:00.272170 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb25db1-43cb-4731-bb5c-095293bcf092 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:00.289423 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aee3bf5-bc48-4bf2-acce-b6bd609e4244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:00.290086 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aee3bf5-bc48-4bf2-acce-b6bd609e4244 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:00.290376 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aee3bf5-bc48-4bf2-acce-b6bd609e4244 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:00.290660 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aee3bf5-bc48-4bf2-acce-b6bd609e4244 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:00.291385 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aee3bf5-bc48-4bf2-acce-b6bd609e4244 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:00.291730 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aee3bf5-bc48-4bf2-acce-b6bd609e4244 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:00.293096 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aee3bf5-bc48-4bf2-acce-b6bd609e4244 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:00.305639 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 240/502] 10.222.0.99 () {56 vars in 1109 bytes} [Thu Jul 14 15:23:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:00.306876 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87296a8c-fd9e-4468-a758-8e6d1c8bf6eb None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:00.307170 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87296a8c-fd9e-4468-a758-8e6d1c8bf6eb None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:00.307453 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87296a8c-fd9e-4468-a758-8e6d1c8bf6eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:00.323276 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-87296a8c-fd9e-4468-a758-8e6d1c8bf6eb None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:23:00.324518 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-87296a8c-fd9e-4468-a758-8e6d1c8bf6eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:23:00.347821 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:00.373299 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 263/503] 10.222.0.99 () {58 vars in 1282 bytes} [Thu Jul 14 15:23:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 15:23:00.374674 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb25db1-43cb-4731-bb5c-095293bcf092 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:00.393673 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 241/504] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:23:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:23:00.394526 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87296a8c-fd9e-4468-a758-8e6d1c8bf6eb None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:00.445562 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:00.477933 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb25db1-43cb-4731-bb5c-095293bcf092 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:00.496603 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87296a8c-fd9e-4468-a758-8e6d1c8bf6eb None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:00.512803 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8ade86-a7fa-435b-9c95-79e6d234f2c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:00.513721 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8ade86-a7fa-435b-9c95-79e6d234f2c8 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:00.514001 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8ade86-a7fa-435b-9c95-79e6d234f2c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:00.514280 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8ade86-a7fa-435b-9c95-79e6d234f2c8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:00.524415 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8ade86-a7fa-435b-9c95-79e6d234f2c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:23:00.531338 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3536072-8511-4bec-8fd3-33cf2230cc6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:00.532031 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3536072-8511-4bec-8fd3-33cf2230cc6f None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:00.532316 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3536072-8511-4bec-8fd3-33cf2230cc6f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:00.532603 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3536072-8511-4bec-8fd3-33cf2230cc6f None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:00.532737 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-ee8ade86-a7fa-435b-9c95-79e6d234f2c8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 15:23:00.532737 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:00.532737 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:00.532737 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:00.532737 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:00.532737 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:00.532737 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:00.532737 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:00.532737 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:00.532737 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:00.532737 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:00.532737 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:00.532737 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:23:00.532737 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:23:00.532737 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:23:00.532737 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:23:00.533833 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:23:00.533833 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:23:00.533833 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:23:00.533833 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:23:00.533833 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:00.533833 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:00.533833 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:00.533833 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:00.533833 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:00.533833 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:00.533833 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:00.533833 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:00.533833 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:00.533833 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:00.533833 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:00.533833 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:00.539777 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:00.539777 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:00.539777 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:00.539777 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:00.539777 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:23:00.539777 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:23:00.539777 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:23:00.539777 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:23:00.539777 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:23:00.539777 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:23:00.539777 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 15:23:00.539777 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:23:00.540537 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3536072-8511-4bec-8fd3-33cf2230cc6f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:00.540537 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3536072-8511-4bec-8fd3-33cf2230cc6f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:00.540692 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 264/505] 10.222.0.99 () {56 vars in 1102 bytes} [Thu Jul 14 15:23:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:00.540692 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8ade86-a7fa-435b-9c95-79e6d234f2c8 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:00.540833 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3536072-8511-4bec-8fd3-33cf2230cc6f None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:00.551802 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 242/506] 10.222.0.99 () {56 vars in 1109 bytes} [Thu Jul 14 15:23:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:00.552648 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3536072-8511-4bec-8fd3-33cf2230cc6f None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:00.573709 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:00.642667 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8ade86-a7fa-435b-9c95-79e6d234f2c8 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:00.656056 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3536072-8511-4bec-8fd3-33cf2230cc6f None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:00.671783 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:00.679175 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fb5c72-c841-4fbf-9497-330828145d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:00.680095 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fb5c72-c841-4fbf-9497-330828145d8b None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:00.680454 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fb5c72-c841-4fbf-9497-330828145d8b None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:00.680765 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fb5c72-c841-4fbf-9497-330828145d8b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:00.688578 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fb5c72-c841-4fbf-9497-330828145d8b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:00.690048 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fb5c72-c841-4fbf-9497-330828145d8b None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:00.690162 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed26fa0d-6d65-4473-bd74-4160409ad470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:00.690924 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed26fa0d-6d65-4473-bd74-4160409ad470 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:00.691208 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed26fa0d-6d65-4473-bd74-4160409ad470 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:00.691481 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed26fa0d-6d65-4473-bd74-4160409ad470 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:00.702948 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed26fa0d-6d65-4473-bd74-4160409ad470 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:00.704410 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed26fa0d-6d65-4473-bd74-4160409ad470 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:00.706318 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-41fb5c72-c841-4fbf-9497-330828145d8b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 14 15:23:00.706318 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:00.706318 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:00.706318 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:00.706318 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:00.706318 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:00.706318 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:00.706318 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:00.706318 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:00.706318 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:00.706318 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:00.706318 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:00.706318 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 15:23:00.706318 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 15:23:00.706318 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 15:23:00.706318 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 15:23:00.707367 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:00.707367 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:00.707367 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 15:23:00.707367 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:23:00.707367 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 15:23:00.707367 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:23:00.707367 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:00.707367 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:00.707367 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:00.707367 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:00.707367 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:00.707367 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:00.707367 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:00.707367 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:00.707367 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:00.707367 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:00.707367 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:00.707367 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:00.708573 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:00.708573 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:00.708573 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 15:23:00.708573 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 15:23:00.708573 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 15:23:00.708573 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 15:23:00.708573 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 15:23:00.708573 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 15:23:00.708573 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 14 15:23:00.708573 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:23:00.708573 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 265/507] 10.222.0.99 () {56 vars in 1093 bytes} [Thu Jul 14 15:23:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:00.709272 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fb5c72-c841-4fbf-9497-330828145d8b None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:00.725761 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-ed26fa0d-6d65-4473-bd74-4160409ad470 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:23:00.725761 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:00.725761 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:00.725761 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:00.725761 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:00.725761 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:00.725761 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:00.725761 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:00.725761 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:00.725761 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:00.725761 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:00.725761 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:00.725761 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:23:00.725761 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:23:00.725761 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:23:00.726816 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:23:00.726816 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:00.726816 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:00.726816 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:00.726816 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:00.726816 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:00.726816 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:00.726816 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:00.726816 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:00.726816 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:00.726816 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:00.726816 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:00.726816 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:00.726816 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:00.726816 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:00.726816 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:00.726816 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:00.727866 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:23:00.727866 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:23:00.727866 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:23:00.727866 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:23:00.727866 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 243/508] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:23:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:00.727866 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed26fa0d-6d65-4473-bd74-4160409ad470 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:00.798584 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:00.813724 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fb5c72-c841-4fbf-9497-330828145d8b None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:00.829881 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed26fa0d-6d65-4473-bd74-4160409ad470 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:00.849426 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2742e5a3-5b3f-4d19-a495-562b460efba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:00.850223 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2742e5a3-5b3f-4d19-a495-562b460efba5 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:00.850508 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2742e5a3-5b3f-4d19-a495-562b460efba5 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:00.850786 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2742e5a3-5b3f-4d19-a495-562b460efba5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:00.861275 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2742e5a3-5b3f-4d19-a495-562b460efba5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:00.862763 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2742e5a3-5b3f-4d19-a495-562b460efba5 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:00.864283 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a26eff1-3125-47ea-b2c4-9bfff5184cb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:00.864956 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a26eff1-3125-47ea-b2c4-9bfff5184cb9 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:00.865241 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a26eff1-3125-47ea-b2c4-9bfff5184cb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:00.865521 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a26eff1-3125-47ea-b2c4-9bfff5184cb9 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:00.866268 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a26eff1-3125-47ea-b2c4-9bfff5184cb9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:00.866579 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a26eff1-3125-47ea-b2c4-9bfff5184cb9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:00.867963 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a26eff1-3125-47ea-b2c4-9bfff5184cb9 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:00.875914 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 244/509] 10.222.0.99 () {56 vars in 1103 bytes} [Thu Jul 14 15:23:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:00.876704 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a26eff1-3125-47ea-b2c4-9bfff5184cb9 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:00.880448 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-2742e5a3-5b3f-4d19-a495-562b460efba5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:23:00.880448 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:00.880448 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:00.880448 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:00.880448 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:00.880448 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:00.880448 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:00.880448 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:00.880448 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:00.880448 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:00.880448 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:00.880448 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:00.880448 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:23:00.880448 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:23:00.880448 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:23:00.881479 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:23:00.881479 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:00.881479 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:00.881479 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:00.881479 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:00.881479 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:00.881479 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:00.881479 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:00.881479 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:00.881479 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:00.881479 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:00.881479 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:00.881479 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:00.881479 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:00.881479 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:00.881479 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:00.881479 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:00.882675 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:23:00.882675 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:23:00.882675 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:23:00.882675 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:23:00.882675 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 266/510] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:23:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:00.883005 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2742e5a3-5b3f-4d19-a495-562b460efba5 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:00.928582 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:00.987388 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a26eff1-3125-47ea-b2c4-9bfff5184cb9 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:00.990965 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2742e5a3-5b3f-4d19-a495-562b460efba5 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:01.020512 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:01.026812 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9a9524-8063-4d6e-ba66-a022d413230d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:01.026965 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab1399c-07d0-4727-896b-8a02b0bdd38d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:01.027499 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9a9524-8063-4d6e-ba66-a022d413230d None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:01.027753 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab1399c-07d0-4727-896b-8a02b0bdd38d None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:01.027846 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9a9524-8063-4d6e-ba66-a022d413230d None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:01.028044 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab1399c-07d0-4727-896b-8a02b0bdd38d None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:01.028142 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9a9524-8063-4d6e-ba66-a022d413230d None admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:01.028326 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab1399c-07d0-4727-896b-8a02b0bdd38d None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:01.028977 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9a9524-8063-4d6e-ba66-a022d413230d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:01.029298 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9a9524-8063-4d6e-ba66-a022d413230d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:01.030735 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9a9524-8063-4d6e-ba66-a022d413230d None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:01.036119 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab1399c-07d0-4727-896b-8a02b0bdd38d None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:01.037542 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab1399c-07d0-4727-896b-8a02b0bdd38d None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:01.049126 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 267/511] 10.222.0.99 () {56 vars in 1133 bytes} [Thu Jul 14 15:23:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:01.049971 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9a9524-8063-4d6e-ba66-a022d413230d None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:01.052412 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-0ab1399c-07d0-4727-896b-8a02b0bdd38d None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 14 15:23:01.052412 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:01.052412 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:01.052412 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:01.052412 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:01.052412 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:01.052412 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:01.052412 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:01.052412 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:01.052412 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:01.052412 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:01.052412 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:01.052412 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 15:23:01.052412 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 15:23:01.052412 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 15:23:01.052412 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 15:23:01.053296 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:01.053296 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:01.053296 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 15:23:01.053296 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:23:01.053296 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 15:23:01.053296 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:23:01.053296 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:01.053296 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:01.053296 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:01.053296 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:01.053296 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:01.053296 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:01.053296 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:01.053296 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:01.053296 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:01.053296 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:01.053296 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:01.053296 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:01.054160 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:01.054160 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:01.054160 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 15:23:01.054160 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 15:23:01.054160 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 15:23:01.054160 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 15:23:01.054160 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 15:23:01.054160 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 15:23:01.054160 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 14 15:23:01.054160 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:23:01.054160 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 245/512] 10.222.0.99 () {56 vars in 1102 bytes} [Thu Jul 14 15:23:00 2022] GET /identity/v3/users/neutron => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:01.054736 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab1399c-07d0-4727-896b-8a02b0bdd38d None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:01.159069 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab1399c-07d0-4727-896b-8a02b0bdd38d None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:01.162348 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:01.165304 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9a9524-8063-4d6e-ba66-a022d413230d None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:01.194008 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a3368e-a968-49ab-a0e3-953daf5111bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:01.194808 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a3368e-a968-49ab-a0e3-953daf5111bf None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:01.195095 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a3368e-a968-49ab-a0e3-953daf5111bf None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:01.195373 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a3368e-a968-49ab-a0e3-953daf5111bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:01.199995 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b050f3e4-075d-45d1-812a-1d3c4b229a35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:01.201374 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b050f3e4-075d-45d1-812a-1d3c4b229a35 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:01.202087 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b050f3e4-075d-45d1-812a-1d3c4b229a35 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:01.202370 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b050f3e4-075d-45d1-812a-1d3c4b229a35 None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:01.203099 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b050f3e4-075d-45d1-812a-1d3c4b229a35 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:01.203408 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b050f3e4-075d-45d1-812a-1d3c4b229a35 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:01.204833 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b050f3e4-075d-45d1-812a-1d3c4b229a35 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:01.206842 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a3368e-a968-49ab-a0e3-953daf5111bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:01.208339 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a3368e-a968-49ab-a0e3-953daf5111bf None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:01.216077 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 268/513] 10.222.0.99 () {56 vars in 1109 bytes} [Thu Jul 14 15:23:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:01.217451 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b050f3e4-075d-45d1-812a-1d3c4b229a35 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:01.230267 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-d1a3368e-a968-49ab-a0e3-953daf5111bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:23:01.230267 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:01.230267 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:01.230267 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:01.230267 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:01.230267 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:01.230267 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:01.230267 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:01.230267 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:01.230267 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:01.230267 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:01.230267 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:01.230267 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:23:01.230267 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:23:01.230267 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:23:01.231128 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:23:01.231128 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:01.231128 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:01.231128 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:01.231128 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:01.231128 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:01.231128 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:01.231128 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:01.231128 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:01.231128 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:01.231128 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:01.231128 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:01.231128 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:01.231128 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:01.231128 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:01.231128 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:01.231128 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:01.232239 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:23:01.232239 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:23:01.232239 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:23:01.232239 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:23:01.232239 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 246/514] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:23:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:01.233284 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a3368e-a968-49ab-a0e3-953daf5111bf None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:01.280258 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:01.325567 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b050f3e4-075d-45d1-812a-1d3c4b229a35 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:01.336692 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a3368e-a968-49ab-a0e3-953daf5111bf None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:01.359532 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caee6a4e-0b68-4477-8a54-025270fa9d4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:01.360234 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caee6a4e-0b68-4477-8a54-025270fa9d4c None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:01.360526 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caee6a4e-0b68-4477-8a54-025270fa9d4c None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:01.360809 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caee6a4e-0b68-4477-8a54-025270fa9d4c None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:01.361529 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caee6a4e-0b68-4477-8a54-025270fa9d4c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:01.361846 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caee6a4e-0b68-4477-8a54-025270fa9d4c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:01.363245 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caee6a4e-0b68-4477-8a54-025270fa9d4c None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:01.370959 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcd969a-c629-4afa-8095-dfb0abb3b461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:01.371919 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcd969a-c629-4afa-8095-dfb0abb3b461 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:01.373110 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcd969a-c629-4afa-8095-dfb0abb3b461 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:01.373556 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcd969a-c629-4afa-8095-dfb0abb3b461 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:01.377682 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 269/515] 10.222.0.99 () {56 vars in 1109 bytes} [Thu Jul 14 15:23:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:01.378561 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caee6a4e-0b68-4477-8a54-025270fa9d4c None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:01.399972 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:01.445258 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcd969a-c629-4afa-8095-dfb0abb3b461 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:01.446887 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcd969a-c629-4afa-8095-dfb0abb3b461 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:01.484168 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caee6a4e-0b68-4477-8a54-025270fa9d4c None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:01.492979 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:01.518942 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac18f79-d73a-4bea-8143-02ab269701c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:01.519699 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac18f79-d73a-4bea-8143-02ab269701c4 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:01.519995 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac18f79-d73a-4bea-8143-02ab269701c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:01.520276 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac18f79-d73a-4bea-8143-02ab269701c4 None admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:01.521124 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac18f79-d73a-4bea-8143-02ab269701c4 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:01.521434 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac18f79-d73a-4bea-8143-02ab269701c4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:01.522930 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac18f79-d73a-4bea-8143-02ab269701c4 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:01.540297 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 270/516] 10.222.0.99 () {56 vars in 1139 bytes} [Thu Jul 14 15:23:01 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:01.541132 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac18f79-d73a-4bea-8143-02ab269701c4 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:01.552809 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 247/517] 10.222.0.99 () {58 vars in 1282 bytes} [Thu Jul 14 15:23:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 15:23:01.553731 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcd969a-c629-4afa-8095-dfb0abb3b461 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:01.620211 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:01.644212 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac18f79-d73a-4bea-8143-02ab269701c4 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:01.660049 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcd969a-c629-4afa-8095-dfb0abb3b461 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:01.680014 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ae278c-8aa7-4708-a418-55cee609c422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:01.680845 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ae278c-8aa7-4708-a418-55cee609c422 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:01.681135 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ae278c-8aa7-4708-a418-55cee609c422 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:01.681416 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ae278c-8aa7-4708-a418-55cee609c422 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:01.689877 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ae278c-8aa7-4708-a418-55cee609c422 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:01.691391 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ae278c-8aa7-4708-a418-55cee609c422 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:01.696756 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074d5e46-9960-4f22-9b53-63079ed4b535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:01.697431 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074d5e46-9960-4f22-9b53-63079ed4b535 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:01.697729 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074d5e46-9960-4f22-9b53-63079ed4b535 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:01.698008 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074d5e46-9960-4f22-9b53-63079ed4b535 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:01.698737 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074d5e46-9960-4f22-9b53-63079ed4b535 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:01.699049 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074d5e46-9960-4f22-9b53-63079ed4b535 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:01.700450 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074d5e46-9960-4f22-9b53-63079ed4b535 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:01.710076 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-c1ae278c-8aa7-4708-a418-55cee609c422 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:23:01.710076 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:01.710076 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:01.710076 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:01.710076 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:01.710076 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:01.710076 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:01.710076 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:01.710076 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:01.710076 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:01.710076 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:01.710076 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:01.710076 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 15:23:01.710076 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 15:23:01.710076 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 15:23:01.711112 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 15:23:01.711112 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:01.711112 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:01.711112 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:01.711112 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:01.711112 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:01.711112 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:01.711112 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:01.711112 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:01.711112 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:01.711112 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:01.711112 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:01.711112 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:01.711112 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:01.711112 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:01.711112 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:01.711112 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:01.712226 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 15:23:01.712226 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 15:23:01.712226 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 15:23:01.712226 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 15:23:01.712226 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 15:23:01.712226 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 15:23:01.712226 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:23:01.712226 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:23:01.712226 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 271/518] 10.222.0.99 () {56 vars in 1111 bytes} [Thu Jul 14 15:23:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:01.712226 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ae278c-8aa7-4708-a418-55cee609c422 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:01.718442 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 248/519] 10.222.0.99 () {56 vars in 1109 bytes} [Thu Jul 14 15:23:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:01.718442 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074d5e46-9960-4f22-9b53-63079ed4b535 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:01.719630 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:01.821354 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074d5e46-9960-4f22-9b53-63079ed4b535 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:01.824486 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ae278c-8aa7-4708-a418-55cee609c422 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:01.858580 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d1f0f0-56ae-4eef-84ed-53c23e77ab2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:01.859319 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d1f0f0-56ae-4eef-84ed-53c23e77ab2d None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:01.859643 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d1f0f0-56ae-4eef-84ed-53c23e77ab2d None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:01.859919 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d1f0f0-56ae-4eef-84ed-53c23e77ab2d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:01.860555 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1cdcf7-59ba-4d25-b62f-a244a3dab1c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:01.862263 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1cdcf7-59ba-4d25-b62f-a244a3dab1c3 None admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:01.862767 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1cdcf7-59ba-4d25-b62f-a244a3dab1c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:01.863245 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1cdcf7-59ba-4d25-b62f-a244a3dab1c3 None admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:01.864900 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1cdcf7-59ba-4d25-b62f-a244a3dab1c3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:01.865954 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1cdcf7-59ba-4d25-b62f-a244a3dab1c3 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:01.875634 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:01.887187 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 272/520] 10.222.0.99 () {60 vars in 1132 bytes} [Thu Jul 14 15:23:01 2022] POST /identity/v3/users => generated 247 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:23:01.888104 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1cdcf7-59ba-4d25-b62f-a244a3dab1c3 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:01.938435 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d1f0f0-56ae-4eef-84ed-53c23e77ab2d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:01.940121 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d1f0f0-56ae-4eef-84ed-53c23e77ab2d None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:01.944868 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:02.035650 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1cdcf7-59ba-4d25-b62f-a244a3dab1c3 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:02.072053 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df51eff5-b127-424a-8964-8ab712396cf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:02.072864 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df51eff5-b127-424a-8964-8ab712396cf2 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:02.073152 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df51eff5-b127-424a-8964-8ab712396cf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:02.073432 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df51eff5-b127-424a-8964-8ab712396cf2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:02.087799 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 249/521] 10.222.0.99 () {58 vars in 1282 bytes} [Thu Jul 14 15:23:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 15:23:02.088682 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d1f0f0-56ae-4eef-84ed-53c23e77ab2d None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:02.090775 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df51eff5-b127-424a-8964-8ab712396cf2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:02.093418 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df51eff5-b127-424a-8964-8ab712396cf2 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:02.115375 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-df51eff5-b127-424a-8964-8ab712396cf2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:23:02.115375 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:02.115375 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:02.115375 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:02.115375 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:02.115375 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:02.115375 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:02.115375 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:02.115375 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:02.115375 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:02.115375 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:02.115375 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:02.115375 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:23:02.115375 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:23:02.115375 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:23:02.116319 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:23:02.116319 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:02.116319 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:02.116319 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:02.116319 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:02.116319 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:02.116319 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:02.116319 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:02.116319 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:02.116319 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:02.116319 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:02.116319 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:02.116319 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:02.116319 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:02.116319 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:02.116319 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:02.116319 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:02.117152 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:23:02.117152 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:23:02.117152 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:23:02.117152 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:23:02.117152 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 273/522] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:23:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:02.117862 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df51eff5-b127-424a-8964-8ab712396cf2 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:02.178964 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:02.207982 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d1f0f0-56ae-4eef-84ed-53c23e77ab2d None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:02.211158 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:02.241921 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df51eff5-b127-424a-8964-8ab712396cf2 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:02.249902 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9fa136-11c3-41b8-970c-dda98cd737cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:02.250573 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9fa136-11c3-41b8-970c-dda98cd737cb None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:02.250857 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9fa136-11c3-41b8-970c-dda98cd737cb None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:02.251134 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9fa136-11c3-41b8-970c-dda98cd737cb None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:02.252306 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9fa136-11c3-41b8-970c-dda98cd737cb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:02.252622 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9fa136-11c3-41b8-970c-dda98cd737cb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:02.254630 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9fa136-11c3-41b8-970c-dda98cd737cb None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:02.279916 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f0e9c7-246e-4142-93c0-d414a557fbab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:02.280850 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f0e9c7-246e-4142-93c0-d414a557fbab None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:02.281138 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f0e9c7-246e-4142-93c0-d414a557fbab None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:02.281418 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f0e9c7-246e-4142-93c0-d414a557fbab None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:02.283388 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 250/523] 10.222.0.99 () {56 vars in 1148 bytes} [Thu Jul 14 15:23:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:02.283932 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9fa136-11c3-41b8-970c-dda98cd737cb None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:02.289646 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f0e9c7-246e-4142-93c0-d414a557fbab None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:02.291145 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f0e9c7-246e-4142-93c0-d414a557fbab None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:02.306609 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-01f0e9c7-246e-4142-93c0-d414a557fbab None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 14 15:23:02.306609 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:02.306609 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:02.306609 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:02.306609 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:02.306609 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:02.306609 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:02.306609 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:02.306609 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:02.306609 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:02.306609 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:02.306609 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:02.306609 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 15:23:02.306609 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 15:23:02.306609 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 15:23:02.306609 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 15:23:02.307570 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:02.307570 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:02.307570 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 15:23:02.307570 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:23:02.307570 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 15:23:02.307570 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:23:02.307570 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:02.307570 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:02.307570 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:02.307570 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:02.307570 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:02.307570 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:02.307570 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:02.307570 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:02.307570 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:02.307570 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:02.307570 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:02.307570 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:02.308622 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:02.308622 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:02.308622 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 15:23:02.308622 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 15:23:02.308622 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 15:23:02.308622 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 15:23:02.308622 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 15:23:02.308622 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 15:23:02.308622 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 14 15:23:02.308622 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:23:02.308622 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 274/524] 10.222.0.99 () {56 vars in 1099 bytes} [Thu Jul 14 15:23:02 2022] GET /identity/v3/users/cinder => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:02.308622 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f0e9c7-246e-4142-93c0-d414a557fbab None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:02.402986 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9fa136-11c3-41b8-970c-dda98cd737cb None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:02.436021 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f0e9c7-246e-4142-93c0-d414a557fbab None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:02.443340 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:02.450247 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51fed96-8a7e-4137-94fa-b805f8d718d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:02.451652 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51fed96-8a7e-4137-94fa-b805f8d718d0 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:02.451952 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51fed96-8a7e-4137-94fa-b805f8d718d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:02.452234 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51fed96-8a7e-4137-94fa-b805f8d718d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:02.462786 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51fed96-8a7e-4137-94fa-b805f8d718d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:02.464295 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51fed96-8a7e-4137-94fa-b805f8d718d0 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:02.475946 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab079b8b-1ccc-47ee-8dfe-388f56999251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:02.476629 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab079b8b-1ccc-47ee-8dfe-388f56999251 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:02.476923 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab079b8b-1ccc-47ee-8dfe-388f56999251 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:02.477210 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab079b8b-1ccc-47ee-8dfe-388f56999251 None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:02.477948 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab079b8b-1ccc-47ee-8dfe-388f56999251 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:02.478263 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab079b8b-1ccc-47ee-8dfe-388f56999251 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:02.479667 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab079b8b-1ccc-47ee-8dfe-388f56999251 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:02.483530 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-e51fed96-8a7e-4137-94fa-b805f8d718d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:23:02.483530 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:02.483530 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:02.483530 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:02.483530 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:02.483530 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:02.483530 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:02.483530 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:02.483530 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:02.483530 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:02.483530 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:02.483530 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:02.483530 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:23:02.483530 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:23:02.483530 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:23:02.484415 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:23:02.484415 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:02.484415 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:02.484415 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:02.484415 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:02.484415 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:02.484415 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:02.484415 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:02.484415 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:02.484415 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:02.484415 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:02.484415 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:02.484415 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:02.484415 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:02.484415 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:02.484415 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:02.484415 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:02.490151 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:23:02.490151 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:23:02.490151 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:23:02.490151 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:23:02.490151 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 251/525] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:23:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:02.490151 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51fed96-8a7e-4137-94fa-b805f8d718d0 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:02.492427 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 275/526] 10.222.0.99 () {56 vars in 1109 bytes} [Thu Jul 14 15:23:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:02.493196 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab079b8b-1ccc-47ee-8dfe-388f56999251 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:02.516231 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:02.603110 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51fed96-8a7e-4137-94fa-b805f8d718d0 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:02.627000 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab079b8b-1ccc-47ee-8dfe-388f56999251 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:02.641639 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142e6f86-1fd8-4951-a5e8-8cc6078abcc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:02.642308 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142e6f86-1fd8-4951-a5e8-8cc6078abcc8 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:02.642600 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142e6f86-1fd8-4951-a5e8-8cc6078abcc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:02.642886 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142e6f86-1fd8-4951-a5e8-8cc6078abcc8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:02.643773 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142e6f86-1fd8-4951-a5e8-8cc6078abcc8 None admin] RBAC: Adding query filter params (role.id=74e3b597938f49d095cb424c54b5a811, scope.project.id=9048d1a6f00443ec92f11471babe5a56, user.id=605e230d50ce44629f554c95b2f79ead) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:02.644091 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142e6f86-1fd8-4951-a5e8-8cc6078abcc8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:02.645548 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142e6f86-1fd8-4951-a5e8-8cc6078abcc8 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:02.651910 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 252/527] 10.222.0.99 () {56 vars in 1374 bytes} [Thu Jul 14 15:23:02 2022] GET /identity/v3/role_assignments?user.id=605e230d50ce44629f554c95b2f79ead&scope.project.id=9048d1a6f00443ec92f11471babe5a56&role.id=74e3b597938f49d095cb424c54b5a811 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:02.652638 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142e6f86-1fd8-4951-a5e8-8cc6078abcc8 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:02.682643 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a0cab0-223a-4732-a6fa-04e6425d5fe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:02.684657 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a0cab0-223a-4732-a6fa-04e6425d5fe9 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:02.685805 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a0cab0-223a-4732-a6fa-04e6425d5fe9 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:02.686391 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a0cab0-223a-4732-a6fa-04e6425d5fe9 None admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:02.687844 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a0cab0-223a-4732-a6fa-04e6425d5fe9 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:02.688862 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a0cab0-223a-4732-a6fa-04e6425d5fe9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:02.691117 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a0cab0-223a-4732-a6fa-04e6425d5fe9 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:02.716319 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 276/528] 10.222.0.99 () {56 vars in 1137 bytes} [Thu Jul 14 15:23:02 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:02.718133 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a0cab0-223a-4732-a6fa-04e6425d5fe9 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:02.754245 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:02.758325 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:02.784950 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142e6f86-1fd8-4951-a5e8-8cc6078abcc8 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:02.834049 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7bfdc7-8c55-4ae3-b10a-de34ea5c9550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:02.834732 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7bfdc7-8c55-4ae3-b10a-de34ea5c9550 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:02.835026 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7bfdc7-8c55-4ae3-b10a-de34ea5c9550 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:02.835312 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7bfdc7-8c55-4ae3-b10a-de34ea5c9550 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:02.836106 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7bfdc7-8c55-4ae3-b10a-de34ea5c9550 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:02.836423 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7bfdc7-8c55-4ae3-b10a-de34ea5c9550 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:02.837821 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7bfdc7-8c55-4ae3-b10a-de34ea5c9550 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:02.850597 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a0cab0-223a-4732-a6fa-04e6425d5fe9 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:02.854371 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 253/529] 10.222.0.99 () {56 vars in 1109 bytes} [Thu Jul 14 15:23:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:02.855362 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7bfdc7-8c55-4ae3-b10a-de34ea5c9550 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:02.892673 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af79707-a480-419e-8f2d-90b39d261a1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:02.893771 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af79707-a480-419e-8f2d-90b39d261a1c None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:02.894065 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af79707-a480-419e-8f2d-90b39d261a1c None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:02.894351 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af79707-a480-419e-8f2d-90b39d261a1c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:02.904431 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af79707-a480-419e-8f2d-90b39d261a1c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:02.905908 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af79707-a480-419e-8f2d-90b39d261a1c None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:02.924404 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-5af79707-a480-419e-8f2d-90b39d261a1c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:23:02.924404 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:02.924404 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:02.924404 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:02.924404 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:02.924404 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:02.924404 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:02.924404 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:02.924404 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:02.924404 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:02.924404 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:02.924404 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:02.924404 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 15:23:02.924404 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 15:23:02.924404 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 15:23:02.925477 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 15:23:02.925477 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:02.925477 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:02.925477 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:02.925477 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:02.925477 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:02.925477 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:02.925477 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:02.925477 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:02.925477 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:02.925477 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:02.925477 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:02.925477 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:02.925477 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:02.925477 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:02.925477 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:02.925477 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:02.926690 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 15:23:02.926690 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 15:23:02.926690 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 15:23:02.926690 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 15:23:02.926690 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 15:23:02.926690 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 15:23:02.926690 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:23:02.926690 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:23:02.926690 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 277/530] 10.222.0.99 () {56 vars in 1111 bytes} [Thu Jul 14 15:23:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:02.927237 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af79707-a480-419e-8f2d-90b39d261a1c None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:02.995366 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7bfdc7-8c55-4ae3-b10a-de34ea5c9550 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:03.030199 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:03.044184 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9170f2c-b0ab-4cfd-bb30-8763561daf8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:03.045381 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9170f2c-b0ab-4cfd-bb30-8763561daf8f None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:03.045820 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9170f2c-b0ab-4cfd-bb30-8763561daf8f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:03.046111 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9170f2c-b0ab-4cfd-bb30-8763561daf8f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:03.049893 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af79707-a480-419e-8f2d-90b39d261a1c None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:03.059174 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:03.088009 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789daae3-eb8d-42e9-8e88-674ca4524919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:03.089251 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789daae3-eb8d-42e9-8e88-674ca4524919 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:03.089779 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789daae3-eb8d-42e9-8e88-674ca4524919 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:03.090284 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789daae3-eb8d-42e9-8e88-674ca4524919 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:03.098663 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789daae3-eb8d-42e9-8e88-674ca4524919 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:03.100646 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789daae3-eb8d-42e9-8e88-674ca4524919 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:03.118063 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-789daae3-eb8d-42e9-8e88-674ca4524919 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:23:03.118063 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:03.118063 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:03.118063 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:03.118063 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:03.118063 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:03.118063 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:03.118063 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:03.118063 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:03.118063 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:03.118063 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:03.118063 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:03.118063 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 15:23:03.118063 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 15:23:03.118063 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 15:23:03.118967 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 15:23:03.118967 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:03.118967 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:03.118967 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:03.118967 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:03.118967 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:03.118967 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:03.118967 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:03.118967 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:03.118967 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:03.118967 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:03.118967 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:03.118967 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:03.118967 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:03.118967 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:03.118967 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:03.118967 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:03.120210 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 15:23:03.120210 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 15:23:03.120210 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 15:23:03.120210 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 15:23:03.120210 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 15:23:03.120210 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 15:23:03.120210 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:23:03.120210 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:23:03.122099 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 278/531] 10.222.0.99 () {56 vars in 1111 bytes} [Thu Jul 14 15:23:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:03.123860 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789daae3-eb8d-42e9-8e88-674ca4524919 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:03.128398 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9170f2c-b0ab-4cfd-bb30-8763561daf8f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:03.129300 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9170f2c-b0ab-4cfd-bb30-8763561daf8f None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:03.235910 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789daae3-eb8d-42e9-8e88-674ca4524919 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:03.256592 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 254/532] 10.222.0.99 () {58 vars in 1282 bytes} [Thu Jul 14 15:23:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 15:23:03.257668 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9170f2c-b0ab-4cfd-bb30-8763561daf8f None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:03.286448 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4f733d-c695-4449-87f9-a8f7df437012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:03.287128 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4f733d-c695-4449-87f9-a8f7df437012 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:03.287419 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4f733d-c695-4449-87f9-a8f7df437012 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:03.287728 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4f733d-c695-4449-87f9-a8f7df437012 None admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:03.289205 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4f733d-c695-4449-87f9-a8f7df437012 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:03.289815 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4f733d-c695-4449-87f9-a8f7df437012 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:03.291494 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4f733d-c695-4449-87f9-a8f7df437012 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:03.308737 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:03.319691 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 279/533] 10.222.0.99 () {56 vars in 1148 bytes} [Thu Jul 14 15:23:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:03.320678 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4f733d-c695-4449-87f9-a8f7df437012 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:03.326623 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:03.398938 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9170f2c-b0ab-4cfd-bb30-8763561daf8f None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:03.440313 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4f733d-c695-4449-87f9-a8f7df437012 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:03.448024 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977fe885-978c-4214-bc65-3be17eb3c9cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:03.448898 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977fe885-978c-4214-bc65-3be17eb3c9cd None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:03.449296 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977fe885-978c-4214-bc65-3be17eb3c9cd None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:03.449654 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977fe885-978c-4214-bc65-3be17eb3c9cd None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:03.450540 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977fe885-978c-4214-bc65-3be17eb3c9cd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:03.450981 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977fe885-978c-4214-bc65-3be17eb3c9cd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:03.452934 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977fe885-978c-4214-bc65-3be17eb3c9cd None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:03.471357 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 255/534] 10.222.0.99 () {56 vars in 1148 bytes} [Thu Jul 14 15:23:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:03.472426 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977fe885-978c-4214-bc65-3be17eb3c9cd None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:03.485270 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc072ed-d656-4265-b5aa-c17391020cb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:03.486102 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc072ed-d656-4265-b5aa-c17391020cb2 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:03.486735 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc072ed-d656-4265-b5aa-c17391020cb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:03.487027 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc072ed-d656-4265-b5aa-c17391020cb2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:03.501571 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc072ed-d656-4265-b5aa-c17391020cb2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:03.503554 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc072ed-d656-4265-b5aa-c17391020cb2 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:03.527903 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-efc072ed-d656-4265-b5aa-c17391020cb2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:23:03.527903 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:03.527903 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:03.527903 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:03.527903 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:03.527903 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:03.527903 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:03.527903 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:03.527903 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:03.527903 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:03.527903 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:03.527903 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:03.527903 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:23:03.527903 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:23:03.527903 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:23:03.529683 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:23:03.529683 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:03.529683 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:03.529683 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:03.529683 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:03.529683 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:03.529683 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:03.529683 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:03.529683 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:03.529683 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:03.529683 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:03.529683 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:03.529683 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:03.529683 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:03.529683 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:03.529683 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:03.529683 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:03.531477 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:23:03.531477 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:23:03.531477 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:23:03.531477 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:23:03.531477 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 280/535] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:23:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:03.531477 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc072ed-d656-4265-b5aa-c17391020cb2 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:03.586579 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:03.592425 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977fe885-978c-4214-bc65-3be17eb3c9cd None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:03.611880 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:03.631821 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e9c4e3-5a38-4261-8a33-f5b236507457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:03.633133 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e9c4e3-5a38-4261-8a33-f5b236507457 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:03.633627 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e9c4e3-5a38-4261-8a33-f5b236507457 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:03.634112 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e9c4e3-5a38-4261-8a33-f5b236507457 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:03.635184 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e9c4e3-5a38-4261-8a33-f5b236507457 None admin] RBAC: Adding query filter params (role.id=74e3b597938f49d095cb424c54b5a811, scope.project.id=9048d1a6f00443ec92f11471babe5a56, user.id=3a9b1c2622934c5185f3b5f1ea623bc0) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:03.635747 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e9c4e3-5a38-4261-8a33-f5b236507457 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:03.637500 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e9c4e3-5a38-4261-8a33-f5b236507457 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:03.645381 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 256/536] 10.222.0.99 () {56 vars in 1374 bytes} [Thu Jul 14 15:23:03 2022] GET /identity/v3/role_assignments?user.id=3a9b1c2622934c5185f3b5f1ea623bc0&scope.project.id=9048d1a6f00443ec92f11471babe5a56&role.id=74e3b597938f49d095cb424c54b5a811 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:03.646489 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e9c4e3-5a38-4261-8a33-f5b236507457 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:03.697634 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc072ed-d656-4265-b5aa-c17391020cb2 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:03.776953 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7decc3-6288-4950-b1e2-f823ae585f7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:03.779214 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7decc3-6288-4950-b1e2-f823ae585f7f None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:03.780099 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7decc3-6288-4950-b1e2-f823ae585f7f None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:03.781812 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7decc3-6288-4950-b1e2-f823ae585f7f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:03.782923 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7decc3-6288-4950-b1e2-f823ae585f7f None admin] RBAC: Adding query filter params (role.id=74e3b597938f49d095cb424c54b5a811, scope.project.id=9048d1a6f00443ec92f11471babe5a56, user.id=0d0261759f9743b1a13a760ca9159305) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:03.783851 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7decc3-6288-4950-b1e2-f823ae585f7f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:03.786327 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7decc3-6288-4950-b1e2-f823ae585f7f None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:03.800513 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 281/537] 10.222.0.99 () {56 vars in 1374 bytes} [Thu Jul 14 15:23:03 2022] GET /identity/v3/role_assignments?user.id=0d0261759f9743b1a13a760ca9159305&scope.project.id=9048d1a6f00443ec92f11471babe5a56&role.id=74e3b597938f49d095cb424c54b5a811 => generated 259 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:03.865845 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e9c4e3-5a38-4261-8a33-f5b236507457 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:03.916539 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ba7e2a-5a6d-4143-8f50-3c4c540232f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:03.917252 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ba7e2a-5a6d-4143-8f50-3c4c540232f7 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:03.917552 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ba7e2a-5a6d-4143-8f50-3c4c540232f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:03.917838 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ba7e2a-5a6d-4143-8f50-3c4c540232f7 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:03.918600 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ba7e2a-5a6d-4143-8f50-3c4c540232f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:03.918945 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ba7e2a-5a6d-4143-8f50-3c4c540232f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:03.921059 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ba7e2a-5a6d-4143-8f50-3c4c540232f7 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:03.943709 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 257/538] 10.222.0.99 () {56 vars in 1109 bytes} [Thu Jul 14 15:23:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:03.946357 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7decc3-6288-4950-b1e2-f823ae585f7f None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:04.118862 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b35b34d-b0cb-487b-b5bc-bf7418a26df1 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:04.119191 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b35b34d-b0cb-487b-b5bc-bf7418a26df1 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:04.119390 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b35b34d-b0cb-487b-b5bc-bf7418a26df1 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:04.120968 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 258/539] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:23:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 15:23:04.127792 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147e5bfa-0afb-4010-9b1e-4904ba56282b None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:04.128461 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147e5bfa-0afb-4010-9b1e-4904ba56282b None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:04.128969 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147e5bfa-0afb-4010-9b1e-4904ba56282b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:04.149853 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-147e5bfa-0afb-4010-9b1e-4904ba56282b None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:23:04.151738 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-147e5bfa-0afb-4010-9b1e-4904ba56282b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:23:04.156893 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7decc3-6288-4950-b1e2-f823ae585f7f None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:04.180676 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:04.199965 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:04.230849 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf86b92-bdc0-4a47-bd65-269aefc9125e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:04.232189 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf86b92-bdc0-4a47-bd65-269aefc9125e None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:04.232739 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf86b92-bdc0-4a47-bd65-269aefc9125e None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:04.233224 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf86b92-bdc0-4a47-bd65-269aefc9125e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:04.244273 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf86b92-bdc0-4a47-bd65-269aefc9125e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:04.246240 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf86b92-bdc0-4a47-bd65-269aefc9125e None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:04.264109 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-dcf86b92-bdc0-4a47-bd65-269aefc9125e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:23:04.264109 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:04.264109 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:04.264109 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:04.264109 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:04.264109 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:04.264109 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:04.264109 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:04.264109 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:04.264109 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:04.264109 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:04.264109 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:04.264109 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 15:23:04.264109 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 15:23:04.264109 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 15:23:04.265199 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 15:23:04.265199 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:04.265199 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:04.265199 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:04.265199 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:04.265199 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:04.265199 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:04.265199 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:04.265199 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:04.265199 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:04.265199 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:04.265199 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:04.265199 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:04.265199 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:04.265199 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:04.265199 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:04.265199 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:04.266056 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 15:23:04.266056 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 15:23:04.266056 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 15:23:04.266056 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 15:23:04.266056 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 15:23:04.266056 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 15:23:04.266056 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:23:04.266056 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:23:04.268847 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 282/540] 10.222.0.99 () {56 vars in 1111 bytes} [Thu Jul 14 15:23:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:04.275363 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf86b92-bdc0-4a47-bd65-269aefc9125e None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:04.293350 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 259/541] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:23:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 15:23:04.354116 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147e5bfa-0afb-4010-9b1e-4904ba56282b None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:04.460254 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf86b92-bdc0-4a47-bd65-269aefc9125e None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:04.510162 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147e5bfa-0afb-4010-9b1e-4904ba56282b None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:04.544669 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e731893b-bbe6-4990-acf2-33520a8f03bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:04.545721 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e731893b-bbe6-4990-acf2-33520a8f03bf None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:04.546265 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e731893b-bbe6-4990-acf2-33520a8f03bf None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:04.546774 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e731893b-bbe6-4990-acf2-33520a8f03bf None admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:04.548449 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e731893b-bbe6-4990-acf2-33520a8f03bf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:04.549293 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e731893b-bbe6-4990-acf2-33520a8f03bf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:04.553789 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e731893b-bbe6-4990-acf2-33520a8f03bf None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:04.576319 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 283/542] 10.222.0.99 () {56 vars in 1148 bytes} [Thu Jul 14 15:23:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:23:04.583151 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e731893b-bbe6-4990-acf2-33520a8f03bf None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:04.589644 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c0b366-35bc-4bc0-a138-dc0000da1ffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:04.590471 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c0b366-35bc-4bc0-a138-dc0000da1ffe None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:04.590761 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c0b366-35bc-4bc0-a138-dc0000da1ffe None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:04.591097 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c0b366-35bc-4bc0-a138-dc0000da1ffe None admin] PATH_INFO: `/v3/roles/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:04.614513 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c0b366-35bc-4bc0-a138-dc0000da1ffe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:23:04.627575 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-18c0b366-35bc-4bc0-a138-dc0000da1ffe None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 15:23:04.627575 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:04.627575 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:04.627575 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:04.627575 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:04.627575 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:04.627575 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:04.627575 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:04.627575 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:04.627575 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:04.627575 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:04.627575 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:04.627575 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:23:04.627575 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:23:04.627575 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:23:04.627575 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:23:04.629326 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:23:04.629326 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:23:04.629326 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:23:04.629326 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:23:04.629326 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:04.629326 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:04.629326 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:04.629326 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:04.629326 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:04.629326 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:04.629326 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:04.629326 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:04.629326 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:04.629326 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:04.629326 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:04.629326 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:04.630698 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:04.630698 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:04.630698 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:04.630698 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:04.630698 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:23:04.630698 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:23:04.630698 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:23:04.630698 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:23:04.630698 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:23:04.630698 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:23:04.630698 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 15:23:04.630698 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:23:04.630698 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 260/543] 10.222.0.99 () {56 vars in 1102 bytes} [Thu Jul 14 15:23:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 15:23:04.640667 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c0b366-35bc-4bc0-a138-dc0000da1ffe None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:04.654804 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:04.678221 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:04.772803 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e731893b-bbe6-4990-acf2-33520a8f03bf None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:04.844014 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf4a9f0-598e-4c1e-a7bf-37f97d3580b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:04.844679 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf4a9f0-598e-4c1e-a7bf-37f97d3580b2 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:04.844971 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf4a9f0-598e-4c1e-a7bf-37f97d3580b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:04.845250 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf4a9f0-598e-4c1e-a7bf-37f97d3580b2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:04.846115 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf4a9f0-598e-4c1e-a7bf-37f97d3580b2 None admin] RBAC: Adding query filter params (role.id=74e3b597938f49d095cb424c54b5a811, scope.project.id=9048d1a6f00443ec92f11471babe5a56, user.id=a3e1a2b83f3d4d88ba5084dfaffd22f3) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:04.846474 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf4a9f0-598e-4c1e-a7bf-37f97d3580b2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:04.848125 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf4a9f0-598e-4c1e-a7bf-37f97d3580b2 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:04.854867 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 284/544] 10.222.0.99 () {56 vars in 1374 bytes} [Thu Jul 14 15:23:04 2022] GET /identity/v3/role_assignments?user.id=a3e1a2b83f3d4d88ba5084dfaffd22f3&scope.project.id=9048d1a6f00443ec92f11471babe5a56&role.id=74e3b597938f49d095cb424c54b5a811 => generated 259 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:04.856012 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c0b366-35bc-4bc0-a138-dc0000da1ffe None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:04.903493 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36905533-ef09-435d-b9ea-ae29fd8e87a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:04.904796 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36905533-ef09-435d-b9ea-ae29fd8e87a6 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:04.905281 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36905533-ef09-435d-b9ea-ae29fd8e87a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:04.905705 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36905533-ef09-435d-b9ea-ae29fd8e87a6 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:04.906516 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36905533-ef09-435d-b9ea-ae29fd8e87a6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:04.906835 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36905533-ef09-435d-b9ea-ae29fd8e87a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:04.908329 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36905533-ef09-435d-b9ea-ae29fd8e87a6 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:04.923783 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 261/545] 10.222.0.99 () {56 vars in 1103 bytes} [Thu Jul 14 15:23:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:04.929348 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf4a9f0-598e-4c1e-a7bf-37f97d3580b2 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:04.991963 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bae56f-b708-456c-a945-55899dd5a213 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:04.992288 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bae56f-b708-456c-a945-55899dd5a213 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:04.992546 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bae56f-b708-456c-a945-55899dd5a213 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:04.994249 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 262/546] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:23:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:23:05.009938 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790ebf20-f64a-4949-978f-0e9a8e393b9d None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:05.010279 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790ebf20-f64a-4949-978f-0e9a8e393b9d None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:05.010509 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790ebf20-f64a-4949-978f-0e9a8e393b9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:05.032620 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-790ebf20-f64a-4949-978f-0e9a8e393b9d None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:23:05.033971 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-790ebf20-f64a-4949-978f-0e9a8e393b9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:23:05.106390 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf4a9f0-598e-4c1e-a7bf-37f97d3580b2 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:05.110415 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:05.125343 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 263/547] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:23:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 15:23:05.156481 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790ebf20-f64a-4949-978f-0e9a8e393b9d None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:05.165206 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:05.188136 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e9aa22-836f-4ed0-a7a5-745597f81b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:05.188960 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e9aa22-836f-4ed0-a7a5-745597f81b22 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:05.189246 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e9aa22-836f-4ed0-a7a5-745597f81b22 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:05.189526 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e9aa22-836f-4ed0-a7a5-745597f81b22 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:05.214190 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e9aa22-836f-4ed0-a7a5-745597f81b22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:05.217344 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e9aa22-836f-4ed0-a7a5-745597f81b22 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:05.249266 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-00e9aa22-836f-4ed0-a7a5-745597f81b22 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:23:05.249266 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:05.249266 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:05.249266 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:05.249266 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:05.249266 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:05.249266 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:05.249266 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:05.249266 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:05.249266 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:05.249266 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:05.249266 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:05.249266 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:23:05.249266 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:23:05.249266 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:23:05.250161 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:23:05.250161 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:05.250161 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:05.250161 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:05.250161 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:05.250161 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:05.250161 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:05.250161 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:05.250161 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:05.250161 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:05.250161 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:05.250161 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:05.250161 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:05.250161 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:05.250161 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:05.250161 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:05.250161 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:05.250996 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:23:05.250996 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:23:05.250996 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:23:05.250996 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:23:05.250996 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 285/548] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:23:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:05.262376 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e9aa22-836f-4ed0-a7a5-745597f81b22 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:05.311804 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790ebf20-f64a-4949-978f-0e9a8e393b9d None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:05.351267 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5ec4ad-e1cb-46dd-85cb-925769a55026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:05.352106 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5ec4ad-e1cb-46dd-85cb-925769a55026 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:05.352397 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5ec4ad-e1cb-46dd-85cb-925769a55026 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:05.352683 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5ec4ad-e1cb-46dd-85cb-925769a55026 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:05.372793 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5ec4ad-e1cb-46dd-85cb-925769a55026 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:23:05.389322 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-ab5ec4ad-e1cb-46dd-85cb-925769a55026 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 15:23:05.389322 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:05.389322 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:05.389322 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:05.389322 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:05.389322 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:05.389322 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:05.389322 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:05.389322 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:05.389322 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:05.389322 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:05.389322 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:05.389322 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:23:05.389322 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:23:05.389322 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:23:05.389322 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:23:05.390439 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:23:05.390439 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:23:05.390439 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:23:05.390439 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:23:05.390439 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:05.390439 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:05.390439 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:05.390439 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:05.390439 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:05.390439 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:05.390439 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:05.390439 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:05.390439 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:05.390439 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:05.390439 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:05.390439 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:05.391794 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:05.391794 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:05.391794 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:05.391794 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:05.391794 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:23:05.391794 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:23:05.391794 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:23:05.391794 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:23:05.391794 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:23:05.391794 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:23:05.391794 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 15:23:05.391794 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:23:05.394968 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 264/549] 10.222.0.99 () {56 vars in 1102 bytes} [Thu Jul 14 15:23:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 15:23:05.401430 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5ec4ad-e1cb-46dd-85cb-925769a55026 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:05.443173 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e9aa22-836f-4ed0-a7a5-745597f81b22 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:05.492030 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe36fc5a-00c0-4336-86d8-049a8e8efba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:05.492785 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe36fc5a-00c0-4336-86d8-049a8e8efba1 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:05.492990 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe36fc5a-00c0-4336-86d8-049a8e8efba1 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:05.493270 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe36fc5a-00c0-4336-86d8-049a8e8efba1 None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:05.494029 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe36fc5a-00c0-4336-86d8-049a8e8efba1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:05.494324 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe36fc5a-00c0-4336-86d8-049a8e8efba1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:05.495743 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe36fc5a-00c0-4336-86d8-049a8e8efba1 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:05.538008 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 286/550] 10.222.0.99 () {56 vars in 1109 bytes} [Thu Jul 14 15:23:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:05.539284 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:05.548967 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe36fc5a-00c0-4336-86d8-049a8e8efba1 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:05.567278 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5ec4ad-e1cb-46dd-85cb-925769a55026 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:05.611228 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0295f696-af6c-4ff1-a46c-1e0aba2b0b27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:05.612014 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0295f696-af6c-4ff1-a46c-1e0aba2b0b27 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:05.612308 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0295f696-af6c-4ff1-a46c-1e0aba2b0b27 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:05.612592 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0295f696-af6c-4ff1-a46c-1e0aba2b0b27 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:05.613375 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0295f696-af6c-4ff1-a46c-1e0aba2b0b27 None admin] RBAC: Adding query filter params (name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:05.613685 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0295f696-af6c-4ff1-a46c-1e0aba2b0b27 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:05.615063 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0295f696-af6c-4ff1-a46c-1e0aba2b0b27 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:05.623968 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:05.624944 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 265/551] 10.222.0.99 () {56 vars in 1103 bytes} [Thu Jul 14 15:23:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:05.631404 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0295f696-af6c-4ff1-a46c-1e0aba2b0b27 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:05.733254 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe36fc5a-00c0-4336-86d8-049a8e8efba1 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:05.771530 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffa18de-a7de-4704-8592-5e2b86a86d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:05.772365 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffa18de-a7de-4704-8592-5e2b86a86d06 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:05.772655 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffa18de-a7de-4704-8592-5e2b86a86d06 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:05.772934 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffa18de-a7de-4704-8592-5e2b86a86d06 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:05.780793 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffa18de-a7de-4704-8592-5e2b86a86d06 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:05.782282 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffa18de-a7de-4704-8592-5e2b86a86d06 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:05.804717 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-7ffa18de-a7de-4704-8592-5e2b86a86d06 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 14 15:23:05.804717 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:05.804717 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:05.804717 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:05.804717 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:05.804717 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:05.804717 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:05.804717 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:05.804717 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:05.804717 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:05.804717 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:05.804717 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:05.804717 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 15:23:05.804717 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 15:23:05.804717 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 15:23:05.804717 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 15:23:05.805804 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:05.805804 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:05.805804 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 15:23:05.805804 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:23:05.805804 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 15:23:05.805804 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:23:05.805804 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:05.805804 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:05.805804 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:05.805804 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:05.805804 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:05.805804 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:05.805804 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:05.805804 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:05.805804 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:05.805804 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:05.805804 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:05.805804 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:05.806805 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:05.806805 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:05.806805 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 15:23:05.806805 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 15:23:05.806805 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 15:23:05.806805 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 15:23:05.806805 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 15:23:05.806805 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 15:23:05.806805 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 14 15:23:05.806805 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:23:05.814248 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 287/552] 10.222.0.99 () {56 vars in 1096 bytes} [Thu Jul 14 15:23:05 2022] GET /identity/v3/users/swift => generated 83 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:05.821254 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffa18de-a7de-4704-8592-5e2b86a86d06 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:05.822487 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0295f696-af6c-4ff1-a46c-1e0aba2b0b27 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:05.855473 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:05.866079 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162b0bff-4f60-4f08-9a03-15613e905eab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:05.867355 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162b0bff-4f60-4f08-9a03-15613e905eab None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:05.867668 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162b0bff-4f60-4f08-9a03-15613e905eab None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:05.867950 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162b0bff-4f60-4f08-9a03-15613e905eab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:05.881520 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162b0bff-4f60-4f08-9a03-15613e905eab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:05.883854 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162b0bff-4f60-4f08-9a03-15613e905eab None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:05.918222 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-162b0bff-4f60-4f08-9a03-15613e905eab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:23:05.918222 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:05.918222 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:05.918222 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:05.918222 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:05.918222 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:05.918222 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:05.918222 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:05.918222 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:05.918222 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:05.918222 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:05.918222 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:05.918222 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:23:05.918222 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:23:05.918222 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:23:05.919148 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:23:05.919148 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:05.919148 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:05.919148 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:05.919148 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:05.919148 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:05.919148 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:05.919148 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:05.919148 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:05.919148 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:05.919148 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:05.919148 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:05.919148 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:05.919148 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:05.919148 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:05.919148 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:05.919148 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:05.920563 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:23:05.920563 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:23:05.920563 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:23:05.920563 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:23:05.922807 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 266/553] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:23:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:05.936180 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162b0bff-4f60-4f08-9a03-15613e905eab None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:06.003768 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:06.051963 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffa18de-a7de-4704-8592-5e2b86a86d06 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:06.111687 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349f21fe-77ca-4377-a5e1-db4c74c9a504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:06.113021 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349f21fe-77ca-4377-a5e1-db4c74c9a504 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:06.114428 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349f21fe-77ca-4377-a5e1-db4c74c9a504 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:06.116264 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349f21fe-77ca-4377-a5e1-db4c74c9a504 None admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:06.117408 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349f21fe-77ca-4377-a5e1-db4c74c9a504 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:06.118018 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349f21fe-77ca-4377-a5e1-db4c74c9a504 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:06.119906 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349f21fe-77ca-4377-a5e1-db4c74c9a504 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:06.155638 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 288/554] 10.222.0.99 () {56 vars in 1135 bytes} [Thu Jul 14 15:23:05 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:23:06.165059 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162b0bff-4f60-4f08-9a03-15613e905eab None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:06.166620 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349f21fe-77ca-4377-a5e1-db4c74c9a504 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:06.204341 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefdabe6-731d-4498-8407-543600977be6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:06.205079 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefdabe6-731d-4498-8407-543600977be6 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:06.205375 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefdabe6-731d-4498-8407-543600977be6 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:06.205710 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefdabe6-731d-4498-8407-543600977be6 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:06.206769 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefdabe6-731d-4498-8407-543600977be6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:06.207091 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefdabe6-731d-4498-8407-543600977be6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:06.208567 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefdabe6-731d-4498-8407-543600977be6 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:06.224783 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 267/555] 10.222.0.99 () {56 vars in 1109 bytes} [Thu Jul 14 15:23:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:23:06.233224 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefdabe6-731d-4498-8407-543600977be6 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:06.302578 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:06.313430 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349f21fe-77ca-4377-a5e1-db4c74c9a504 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:06.354611 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7a7581-e628-42d0-b9c1-d3309f50d23e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:06.355840 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7a7581-e628-42d0-b9c1-d3309f50d23e None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:06.356203 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7a7581-e628-42d0-b9c1-d3309f50d23e None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:06.356557 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7a7581-e628-42d0-b9c1-d3309f50d23e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:06.360439 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:06.373973 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefdabe6-731d-4498-8407-543600977be6 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:06.413336 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d48f1f-3147-47fa-8cf1-2da4c03100c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:06.414179 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d48f1f-3147-47fa-8cf1-2da4c03100c5 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:06.414441 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d48f1f-3147-47fa-8cf1-2da4c03100c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:06.414721 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d48f1f-3147-47fa-8cf1-2da4c03100c5 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:06.422784 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d48f1f-3147-47fa-8cf1-2da4c03100c5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:06.424277 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d48f1f-3147-47fa-8cf1-2da4c03100c5 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:06.438853 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7a7581-e628-42d0-b9c1-d3309f50d23e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:06.439450 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-e0d48f1f-3147-47fa-8cf1-2da4c03100c5 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 14 15:23:06.439450 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:06.439450 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:06.439450 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:06.439450 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:06.439450 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:06.439450 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:06.439450 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:06.439450 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:06.439450 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:06.439450 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:06.439450 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:06.439450 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 15:23:06.439450 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 15:23:06.439450 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 15:23:06.439450 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 15:23:06.445868 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:06.445868 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:06.445868 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 15:23:06.445868 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:23:06.445868 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 15:23:06.445868 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:23:06.445868 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:06.445868 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:06.445868 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:06.445868 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:06.445868 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:06.445868 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:06.445868 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:06.445868 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:06.445868 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:06.445868 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:06.445868 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:06.445868 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:06.446795 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:06.446795 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:06.446795 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 15:23:06.446795 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 15:23:06.446795 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 15:23:06.446795 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 15:23:06.446795 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 15:23:06.446795 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 15:23:06.446795 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 14 15:23:06.446795 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:23:06.447313 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7a7581-e628-42d0-b9c1-d3309f50d23e None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:06.447384 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 268/556] 10.222.0.99 () {56 vars in 1099 bytes} [Thu Jul 14 15:23:06 2022] GET /identity/v3/users/glance => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:06.447384 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c850ad3-a543-4be6-bcb6-6ec36962eb55 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:06.447384 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c850ad3-a543-4be6-bcb6-6ec36962eb55 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:06.447384 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c850ad3-a543-4be6-bcb6-6ec36962eb55 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:06.447384 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 269/557] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:23:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:23:06.447384 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcee3ae-5d30-4987-bbff-1b0c2501258d None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:06.447384 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcee3ae-5d30-4987-bbff-1b0c2501258d None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:06.447802 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcee3ae-5d30-4987-bbff-1b0c2501258d None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:06.447802 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 270/558] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:23:06 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:23:06.447933 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcee3ae-5d30-4987-bbff-1b0c2501258d None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:06.557787 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:06.558738 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 289/559] 10.222.0.99 () {58 vars in 1282 bytes} [Thu Jul 14 15:23:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 14 15:23:06.561010 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55186c1c-40f9-4261-845c-03af46239a8c None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:06.561010 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55186c1c-40f9-4261-845c-03af46239a8c None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:06.561273 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55186c1c-40f9-4261-845c-03af46239a8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:06.568608 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcee3ae-5d30-4987-bbff-1b0c2501258d None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:06.579379 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-55186c1c-40f9-4261-845c-03af46239a8c None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:23:06.580852 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-55186c1c-40f9-4261-845c-03af46239a8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:23:06.611546 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:06.618017 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3f36ad-59c6-47aa-9669-f7a43ef8b274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:06.618700 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3f36ad-59c6-47aa-9669-f7a43ef8b274 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:06.618993 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3f36ad-59c6-47aa-9669-f7a43ef8b274 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:06.619279 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3f36ad-59c6-47aa-9669-f7a43ef8b274 None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:06.620166 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3f36ad-59c6-47aa-9669-f7a43ef8b274 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:06.620475 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3f36ad-59c6-47aa-9669-f7a43ef8b274 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:06.621937 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3f36ad-59c6-47aa-9669-f7a43ef8b274 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:06.661700 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 290/560] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:23:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:23:06.667264 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c81df6d-4ed3-49e0-bc83-085819a88c93 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:06.667264 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c81df6d-4ed3-49e0-bc83-085819a88c93 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:06.667264 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c81df6d-4ed3-49e0-bc83-085819a88c93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:06.668039 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 271/561] 10.222.0.99 () {56 vars in 1137 bytes} [Thu Jul 14 15:23:06 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:06.669331 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3f36ad-59c6-47aa-9669-f7a43ef8b274 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:06.684775 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-8c81df6d-4ed3-49e0-bc83-085819a88c93 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:23:06.686166 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-8c81df6d-4ed3-49e0-bc83-085819a88c93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:23:06.783625 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 291/562] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:23:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:23:06.783625 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c81df6d-4ed3-49e0-bc83-085819a88c93 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:06.813127 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3f36ad-59c6-47aa-9669-f7a43ef8b274 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:06.867004 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cb3433-7714-4e12-a599-4e464e4813c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:06.868156 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cb3433-7714-4e12-a599-4e464e4813c4 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:06.868625 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cb3433-7714-4e12-a599-4e464e4813c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:06.868907 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cb3433-7714-4e12-a599-4e464e4813c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:06.879915 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cb3433-7714-4e12-a599-4e464e4813c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:06.881760 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cb3433-7714-4e12-a599-4e464e4813c4 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:06.900583 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:06.903261 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-95cb3433-7714-4e12-a599-4e464e4813c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:23:06.903261 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:06.903261 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:06.903261 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:06.903261 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:06.903261 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:06.903261 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:06.903261 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:06.903261 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:06.903261 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:06.903261 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:06.903261 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:06.903261 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:23:06.903261 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:23:06.903261 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:23:06.905282 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:23:06.905282 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:06.905282 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:06.905282 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:06.905282 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:06.905282 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:06.905282 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:06.905282 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:06.905282 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:06.905282 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:06.905282 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:06.905282 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:06.905282 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:06.905282 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:06.905282 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:06.905282 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:06.905282 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:06.906604 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:23:06.906604 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:23:06.906604 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:23:06.906604 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:23:06.906604 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 272/563] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:23:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:06.906604 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cb3433-7714-4e12-a599-4e464e4813c4 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:06.916691 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c81df6d-4ed3-49e0-bc83-085819a88c93 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:06.922956 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:06.962263 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d45c77-6fda-48eb-b0f2-51ed9e406d8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:06.963163 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d45c77-6fda-48eb-b0f2-51ed9e406d8e None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:06.963420 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d45c77-6fda-48eb-b0f2-51ed9e406d8e None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:06.963674 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d45c77-6fda-48eb-b0f2-51ed9e406d8e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:06.975092 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d45c77-6fda-48eb-b0f2-51ed9e406d8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:06.976676 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d45c77-6fda-48eb-b0f2-51ed9e406d8e None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:07.003532 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-f4d45c77-6fda-48eb-b0f2-51ed9e406d8e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:23:07.003532 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:07.003532 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:07.003532 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:07.003532 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:07.003532 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:07.003532 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:07.003532 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:07.003532 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:07.003532 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:07.003532 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:07.003532 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:07.003532 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:23:07.003532 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:23:07.003532 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:23:07.004464 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:23:07.004464 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:07.004464 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:07.004464 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:07.004464 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:07.004464 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:07.004464 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:07.004464 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:07.004464 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:07.004464 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:07.004464 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:07.004464 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:07.004464 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:07.004464 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:07.004464 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:07.004464 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:07.004464 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:07.010516 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:23:07.010516 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:23:07.010516 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:23:07.010516 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:23:07.010516 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 292/564] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:23:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:07.010516 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d45c77-6fda-48eb-b0f2-51ed9e406d8e None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:07.072358 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cb3433-7714-4e12-a599-4e464e4813c4 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:07.110940 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a14ae7c-d229-43b8-b69f-384f6e04e88a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:07.111786 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a14ae7c-d229-43b8-b69f-384f6e04e88a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:07.112071 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a14ae7c-d229-43b8-b69f-384f6e04e88a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:07.112348 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a14ae7c-d229-43b8-b69f-384f6e04e88a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:07.124881 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a14ae7c-d229-43b8-b69f-384f6e04e88a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:23:07.132486 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d45c77-6fda-48eb-b0f2-51ed9e406d8e None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:07.132712 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-2a14ae7c-d229-43b8-b69f-384f6e04e88a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 15:23:07.132712 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:07.132712 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:07.132712 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:07.132712 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:07.132712 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:07.132712 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:07.132712 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:07.132712 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:07.132712 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:07.132712 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:07.132712 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:07.132712 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:23:07.132712 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:23:07.132712 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:23:07.132712 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:23:07.133858 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:23:07.133858 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:23:07.133858 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:23:07.133858 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:23:07.133858 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:07.133858 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:07.133858 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:07.133858 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:07.133858 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:07.133858 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:07.133858 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:07.133858 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:07.133858 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:07.133858 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:07.133858 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:07.133858 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:07.134683 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:07.134683 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:07.134683 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:07.134683 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:07.134683 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:23:07.134683 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:23:07.134683 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:23:07.134683 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:23:07.134683 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:23:07.134683 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:23:07.134683 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 15:23:07.134683 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:23:07.134683 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 273/565] 10.222.0.99 () {56 vars in 1102 bytes} [Thu Jul 14 15:23:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:07.136117 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a14ae7c-d229-43b8-b69f-384f6e04e88a None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:07.175800 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8425607-9cee-4b33-a784-08bc737f1bc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:07.176541 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8425607-9cee-4b33-a784-08bc737f1bc3 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:07.176834 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8425607-9cee-4b33-a784-08bc737f1bc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:07.177119 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8425607-9cee-4b33-a784-08bc737f1bc3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:07.188797 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8425607-9cee-4b33-a784-08bc737f1bc3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:23:07.196862 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-e8425607-9cee-4b33-a784-08bc737f1bc3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 15:23:07.196862 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:07.196862 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:07.196862 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:07.196862 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:07.196862 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:07.196862 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:07.196862 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:07.196862 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:07.196862 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:07.196862 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:07.196862 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:07.196862 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:23:07.196862 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:23:07.196862 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:23:07.196862 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:23:07.197834 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:23:07.197834 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:23:07.197834 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:23:07.197834 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:23:07.197834 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:07.197834 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:07.197834 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:07.197834 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:07.197834 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:07.197834 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:07.197834 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:07.197834 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:07.197834 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:07.197834 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:07.197834 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:07.197834 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:07.198656 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:07.198656 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:07.198656 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:07.198656 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:07.198656 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:23:07.198656 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:23:07.198656 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:23:07.198656 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:23:07.198656 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:23:07.198656 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:23:07.198656 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 15:23:07.198656 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:23:07.198656 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 293/566] 10.222.0.99 () {56 vars in 1102 bytes} [Thu Jul 14 15:23:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:07.200156 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8425607-9cee-4b33-a784-08bc737f1bc3 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:07.210029 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:07.220676 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:07.256243 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a14ae7c-d229-43b8-b69f-384f6e04e88a None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:07.297008 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d2feb5-e58c-47d9-a283-9a185175d771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:07.297907 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d2feb5-e58c-47d9-a283-9a185175d771 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:07.298360 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d2feb5-e58c-47d9-a283-9a185175d771 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:07.298826 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d2feb5-e58c-47d9-a283-9a185175d771 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:07.300064 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d2feb5-e58c-47d9-a283-9a185175d771 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:07.300586 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d2feb5-e58c-47d9-a283-9a185175d771 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:07.302995 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d2feb5-e58c-47d9-a283-9a185175d771 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:07.316359 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 274/567] 10.222.0.99 () {56 vars in 1109 bytes} [Thu Jul 14 15:23:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:07.317169 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d2feb5-e58c-47d9-a283-9a185175d771 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:07.320969 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8425607-9cee-4b33-a784-08bc737f1bc3 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:07.356982 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eea1865-a451-4589-b2ab-9fd1f685f90e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:07.357718 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eea1865-a451-4589-b2ab-9fd1f685f90e None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:07.358202 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eea1865-a451-4589-b2ab-9fd1f685f90e None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:07.358492 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eea1865-a451-4589-b2ab-9fd1f685f90e None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:07.359246 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eea1865-a451-4589-b2ab-9fd1f685f90e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:07.359560 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eea1865-a451-4589-b2ab-9fd1f685f90e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:07.361018 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eea1865-a451-4589-b2ab-9fd1f685f90e None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:07.379776 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 294/568] 10.222.0.99 () {56 vars in 1109 bytes} [Thu Jul 14 15:23:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:07.380688 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eea1865-a451-4589-b2ab-9fd1f685f90e None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:07.426761 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d2feb5-e58c-47d9-a283-9a185175d771 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:07.448021 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:07.455569 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:07.462843 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61641656-cacd-4386-a7f4-3676eb50eab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:07.463522 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61641656-cacd-4386-a7f4-3676eb50eab1 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:07.463845 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61641656-cacd-4386-a7f4-3676eb50eab1 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:07.464128 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61641656-cacd-4386-a7f4-3676eb50eab1 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:07.464886 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61641656-cacd-4386-a7f4-3676eb50eab1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:07.465475 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61641656-cacd-4386-a7f4-3676eb50eab1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:07.466735 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61641656-cacd-4386-a7f4-3676eb50eab1 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:07.475466 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 275/569] 10.222.0.99 () {56 vars in 1103 bytes} [Thu Jul 14 15:23:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:07.476624 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11269e5-6cf2-4aad-be80-10e0a2340f8e None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:07.476897 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11269e5-6cf2-4aad-be80-10e0a2340f8e None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:07.477181 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11269e5-6cf2-4aad-be80-10e0a2340f8e None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:07.478575 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 276/570] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:23:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:23:07.479149 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11269e5-6cf2-4aad-be80-10e0a2340f8e None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:07.484056 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eea1865-a451-4589-b2ab-9fd1f685f90e None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:07.522207 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b4e7a0-eca5-4553-a11c-abd32a585aea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:07.522891 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b4e7a0-eca5-4553-a11c-abd32a585aea None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:07.523180 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b4e7a0-eca5-4553-a11c-abd32a585aea None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:07.523460 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b4e7a0-eca5-4553-a11c-abd32a585aea None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:07.524248 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b4e7a0-eca5-4553-a11c-abd32a585aea None admin] RBAC: Adding query filter params (name=service) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:07.524554 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b4e7a0-eca5-4553-a11c-abd32a585aea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:07.525938 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b4e7a0-eca5-4553-a11c-abd32a585aea None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:07.534942 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 295/571] 10.222.0.99 () {56 vars in 1103 bytes} [Thu Jul 14 15:23:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:07.535823 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b4e7a0-eca5-4553-a11c-abd32a585aea None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:07.589994 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11269e5-6cf2-4aad-be80-10e0a2340f8e None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:07.625334 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f5bdc2-2cea-4773-9dd9-bd754472c999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:07.626119 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f5bdc2-2cea-4773-9dd9-bd754472c999 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:07.626406 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f5bdc2-2cea-4773-9dd9-bd754472c999 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:07.626689 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f5bdc2-2cea-4773-9dd9-bd754472c999 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:07.633997 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f5bdc2-2cea-4773-9dd9-bd754472c999 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:07.635453 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f5bdc2-2cea-4773-9dd9-bd754472c999 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:07.650605 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-73f5bdc2-2cea-4773-9dd9-bd754472c999 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:23:07.650605 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:07.650605 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:07.650605 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:07.650605 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:07.650605 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:07.650605 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:07.650605 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:07.650605 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:07.650605 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:07.650605 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:07.650605 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:07.650605 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 15:23:07.650605 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 15:23:07.650605 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 15:23:07.651644 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 15:23:07.651644 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:07.651644 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:07.651644 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:07.651644 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:07.651644 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:07.651644 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:07.651644 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:07.651644 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:07.651644 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:07.651644 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:07.651644 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:07.651644 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:07.651644 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:07.651644 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:07.651644 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:07.651644 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:07.652595 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 15:23:07.652595 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 15:23:07.652595 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 15:23:07.652595 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 15:23:07.652595 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 15:23:07.652595 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 15:23:07.652595 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:23:07.652595 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:23:07.652595 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 277/572] 10.222.0.99 () {56 vars in 1111 bytes} [Thu Jul 14 15:23:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:07.653445 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b4e7a0-eca5-4553-a11c-abd32a585aea None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:07.653887 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122a9fed-2c33-4515-a112-ed16da764bdc None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:07.653887 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122a9fed-2c33-4515-a112-ed16da764bdc None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:07.653887 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122a9fed-2c33-4515-a112-ed16da764bdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:07.672566 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-122a9fed-2c33-4515-a112-ed16da764bdc None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:23:07.673794 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-122a9fed-2c33-4515-a112-ed16da764bdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:23:07.683796 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:07.692395 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf3d057-e12f-4980-bae1-f7c33d2ade82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:07.693300 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf3d057-e12f-4980-bae1-f7c33d2ade82 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:07.693937 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf3d057-e12f-4980-bae1-f7c33d2ade82 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:07.693937 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf3d057-e12f-4980-bae1-f7c33d2ade82 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:07.700636 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf3d057-e12f-4980-bae1-f7c33d2ade82 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:07.702616 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf3d057-e12f-4980-bae1-f7c33d2ade82 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:07.716573 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-4bf3d057-e12f-4980-bae1-f7c33d2ade82 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:23:07.716573 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:07.716573 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:07.716573 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:07.716573 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:07.716573 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:07.716573 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:07.716573 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:07.716573 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:07.716573 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:07.716573 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:07.716573 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:07.716573 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 15:23:07.716573 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 15:23:07.716573 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 15:23:07.717767 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 15:23:07.717767 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:07.717767 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:07.717767 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:07.717767 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:07.717767 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:07.717767 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:07.717767 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:07.717767 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:07.717767 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:07.717767 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:07.717767 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:07.717767 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:07.717767 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:07.717767 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:07.717767 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:07.717767 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:07.718653 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 15:23:07.718653 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 15:23:07.718653 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 15:23:07.718653 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 15:23:07.718653 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 15:23:07.718653 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 15:23:07.718653 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:23:07.718653 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:23:07.718653 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 296/573] 10.222.0.99 () {56 vars in 1111 bytes} [Thu Jul 14 15:23:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:07.718653 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf3d057-e12f-4980-bae1-f7c33d2ade82 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:07.725853 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:07.746259 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 278/574] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:23:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:23:07.747114 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122a9fed-2c33-4515-a112-ed16da764bdc None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:07.829798 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf3d057-e12f-4980-bae1-f7c33d2ade82 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:07.857256 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122a9fed-2c33-4515-a112-ed16da764bdc None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:07.868549 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62eaeb9d-c3ef-4ec3-8374-50483c353897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:07.869485 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eaeb9d-c3ef-4ec3-8374-50483c353897 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:07.870316 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eaeb9d-c3ef-4ec3-8374-50483c353897 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:07.870316 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eaeb9d-c3ef-4ec3-8374-50483c353897 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:07.881437 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62eaeb9d-c3ef-4ec3-8374-50483c353897 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:07.883167 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62eaeb9d-c3ef-4ec3-8374-50483c353897 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:07.895832 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c124a965-cefa-4b7c-8d3f-7eff71d6e068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:07.896628 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c124a965-cefa-4b7c-8d3f-7eff71d6e068 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:07.896916 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c124a965-cefa-4b7c-8d3f-7eff71d6e068 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:07.897196 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c124a965-cefa-4b7c-8d3f-7eff71d6e068 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:07.901621 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-62eaeb9d-c3ef-4ec3-8374-50483c353897 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:23:07.901621 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:07.901621 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:07.901621 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:07.901621 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:07.901621 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:07.901621 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:07.901621 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:07.901621 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:07.901621 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:07.901621 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:07.901621 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:07.901621 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:23:07.901621 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:23:07.901621 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:23:07.902703 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:23:07.902703 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:07.902703 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:07.902703 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:07.902703 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:07.902703 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:07.902703 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:07.902703 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:07.902703 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:07.902703 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:07.902703 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:07.902703 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:07.902703 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:07.902703 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:07.902703 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:07.902703 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:07.902703 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:07.903633 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:23:07.903633 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:23:07.903633 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:23:07.903633 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:23:07.903633 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 297/575] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:23:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:07.903937 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62eaeb9d-c3ef-4ec3-8374-50483c353897 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:07.910896 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c124a965-cefa-4b7c-8d3f-7eff71d6e068 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:07.912412 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c124a965-cefa-4b7c-8d3f-7eff71d6e068 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:07.933489 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-c124a965-cefa-4b7c-8d3f-7eff71d6e068 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:23:07.933489 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:07.933489 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:07.933489 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:07.933489 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:07.933489 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:07.933489 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:07.933489 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:07.933489 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:07.933489 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:07.933489 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:07.933489 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:07.933489 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:23:07.933489 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:23:07.933489 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:23:07.934488 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:23:07.934488 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:07.934488 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:07.934488 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:07.934488 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:07.934488 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:07.934488 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:07.934488 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:07.934488 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:07.934488 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:07.934488 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:07.934488 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:07.934488 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:07.934488 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:07.934488 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:07.934488 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:07.934488 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:07.935959 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:23:07.935959 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:23:07.935959 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:23:07.935959 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:23:07.935959 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 279/576] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:23:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:07.936902 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c124a965-cefa-4b7c-8d3f-7eff71d6e068 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:07.954915 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:07.979302 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:08.016782 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62eaeb9d-c3ef-4ec3-8374-50483c353897 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:08.054318 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcea2a63-3699-43ae-a2e6-8784e6bde531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:08.054991 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcea2a63-3699-43ae-a2e6-8784e6bde531 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:08.055276 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcea2a63-3699-43ae-a2e6-8784e6bde531 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:08.055559 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcea2a63-3699-43ae-a2e6-8784e6bde531 None admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:08.056402 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcea2a63-3699-43ae-a2e6-8784e6bde531 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:08.056717 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcea2a63-3699-43ae-a2e6-8784e6bde531 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:08.058162 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcea2a63-3699-43ae-a2e6-8784e6bde531 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:08.062708 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c124a965-cefa-4b7c-8d3f-7eff71d6e068 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:08.071627 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 298/577] 10.222.0.99 () {56 vars in 1148 bytes} [Thu Jul 14 15:23:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:08.072457 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcea2a63-3699-43ae-a2e6-8784e6bde531 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:08.111327 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3bf239-bf7e-4fd2-9225-4eb74e7b0657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:08.112029 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3bf239-bf7e-4fd2-9225-4eb74e7b0657 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:08.112322 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3bf239-bf7e-4fd2-9225-4eb74e7b0657 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:08.112605 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3bf239-bf7e-4fd2-9225-4eb74e7b0657 None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:08.113400 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3bf239-bf7e-4fd2-9225-4eb74e7b0657 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:08.113716 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3bf239-bf7e-4fd2-9225-4eb74e7b0657 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:08.115185 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3bf239-bf7e-4fd2-9225-4eb74e7b0657 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:08.127905 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 280/578] 10.222.0.99 () {56 vars in 1148 bytes} [Thu Jul 14 15:23:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:08.128726 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3bf239-bf7e-4fd2-9225-4eb74e7b0657 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:08.178771 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:08.182182 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcea2a63-3699-43ae-a2e6-8784e6bde531 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:08.217634 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a01b53b-5947-42bf-96b8-22b466b7955b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:08.218372 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a01b53b-5947-42bf-96b8-22b466b7955b None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:08.218685 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a01b53b-5947-42bf-96b8-22b466b7955b None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:08.218969 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a01b53b-5947-42bf-96b8-22b466b7955b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:08.229079 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a01b53b-5947-42bf-96b8-22b466b7955b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:23:08.229813 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:08.237563 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3bf239-bf7e-4fd2-9225-4eb74e7b0657 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:08.237895 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-7a01b53b-5947-42bf-96b8-22b466b7955b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 15:23:08.237895 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:08.237895 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:08.237895 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:08.237895 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:08.237895 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:08.237895 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:08.237895 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:08.237895 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:08.237895 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:08.237895 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:08.237895 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:08.237895 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:23:08.237895 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:23:08.237895 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:23:08.237895 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:23:08.238921 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:23:08.238921 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:23:08.238921 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:23:08.238921 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:23:08.238921 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:08.238921 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:08.238921 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:08.238921 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:08.238921 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:08.238921 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:08.238921 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:08.238921 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:08.238921 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:08.238921 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:08.238921 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:08.238921 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:08.240009 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:08.240009 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:08.240009 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:08.240009 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:08.240009 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:23:08.240009 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:23:08.240009 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:23:08.240009 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:23:08.240009 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:23:08.240009 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:23:08.240009 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 15:23:08.240009 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:23:08.240009 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 299/579] 10.222.0.99 () {56 vars in 1102 bytes} [Thu Jul 14 15:23:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:08.240009 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a01b53b-5947-42bf-96b8-22b466b7955b None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:08.290094 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0458343a-47d7-4a93-86f4-c463a2643b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:08.290765 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0458343a-47d7-4a93-86f4-c463a2643b3f None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:08.291224 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0458343a-47d7-4a93-86f4-c463a2643b3f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:08.291507 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0458343a-47d7-4a93-86f4-c463a2643b3f None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:08.292265 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0458343a-47d7-4a93-86f4-c463a2643b3f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:08.292582 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0458343a-47d7-4a93-86f4-c463a2643b3f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:08.293962 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0458343a-47d7-4a93-86f4-c463a2643b3f None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:08.306762 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 281/580] 10.222.0.99 () {56 vars in 1109 bytes} [Thu Jul 14 15:23:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:08.307571 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0458343a-47d7-4a93-86f4-c463a2643b3f None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:08.344782 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a01b53b-5947-42bf-96b8-22b466b7955b None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:08.378806 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51141c5b-5604-440e-90ea-0d7f5437bd60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:08.379474 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51141c5b-5604-440e-90ea-0d7f5437bd60 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:08.379794 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51141c5b-5604-440e-90ea-0d7f5437bd60 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:08.380074 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51141c5b-5604-440e-90ea-0d7f5437bd60 None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:08.380805 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51141c5b-5604-440e-90ea-0d7f5437bd60 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:08.381114 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51141c5b-5604-440e-90ea-0d7f5437bd60 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:08.382525 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51141c5b-5604-440e-90ea-0d7f5437bd60 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:08.395705 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 300/581] 10.222.0.99 () {56 vars in 1109 bytes} [Thu Jul 14 15:23:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:08.396544 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51141c5b-5604-440e-90ea-0d7f5437bd60 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:08.419874 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0458343a-47d7-4a93-86f4-c463a2643b3f None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:08.422458 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:08.455678 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0341e977-b866-4717-87d8-7256d0727b61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:08.456337 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0341e977-b866-4717-87d8-7256d0727b61 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:08.456624 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0341e977-b866-4717-87d8-7256d0727b61 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:08.456904 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0341e977-b866-4717-87d8-7256d0727b61 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:08.457748 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0341e977-b866-4717-87d8-7256d0727b61 None admin] RBAC: Adding query filter params (role.id=74e3b597938f49d095cb424c54b5a811, scope.project.id=9048d1a6f00443ec92f11471babe5a56, user.id=ddd2ec3b3ebc4aacb322c95270374451) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:08.458062 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0341e977-b866-4717-87d8-7256d0727b61 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:08.459503 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0341e977-b866-4717-87d8-7256d0727b61 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:08.464991 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 282/582] 10.222.0.99 () {56 vars in 1374 bytes} [Thu Jul 14 15:23:08 2022] GET /identity/v3/role_assignments?user.id=ddd2ec3b3ebc4aacb322c95270374451&scope.project.id=9048d1a6f00443ec92f11471babe5a56&role.id=74e3b597938f49d095cb424c54b5a811 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:08.465841 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0341e977-b866-4717-87d8-7256d0727b61 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:08.490129 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:08.537299 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51141c5b-5604-440e-90ea-0d7f5437bd60 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:08.578240 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c9a43a-8daf-4486-bd14-318c6a01a9bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:08.579835 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c9a43a-8daf-4486-bd14-318c6a01a9bf None admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:08.580218 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c9a43a-8daf-4486-bd14-318c6a01a9bf None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:08.580506 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c9a43a-8daf-4486-bd14-318c6a01a9bf None admin] PATH_INFO: `/v3/projects/9048d1a6f00443ec92f11471babe5a56/users/605e230d50ce44629f554c95b2f79ead/roles/74e3b597938f49d095cb424c54b5a811` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:08.588701 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0341e977-b866-4717-87d8-7256d0727b61 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:08.602896 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c9a43a-8daf-4486-bd14-318c6a01a9bf None admin] RBAC: Authorizing `identity:create_grant(project_id=9048d1a6f00443ec92f11471babe5a56, user_id=605e230d50ce44629f554c95b2f79ead, role_id=74e3b597938f49d095cb424c54b5a811)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:08.604481 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c9a43a-8daf-4486-bd14-318c6a01a9bf None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:08.622280 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 301/583] 10.222.0.99 () {58 vars in 1439 bytes} [Thu Jul 14 15:23:08 2022] PUT /identity/v3/projects/9048d1a6f00443ec92f11471babe5a56/users/605e230d50ce44629f554c95b2f79ead/roles/74e3b597938f49d095cb424c54b5a811 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:23:08.623679 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c9a43a-8daf-4486-bd14-318c6a01a9bf None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:08.636226 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d89b57-136f-47cb-bb1a-59ef08672a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:08.636965 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d89b57-136f-47cb-bb1a-59ef08672a81 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:08.637224 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d89b57-136f-47cb-bb1a-59ef08672a81 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:08.637504 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d89b57-136f-47cb-bb1a-59ef08672a81 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:08.638301 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d89b57-136f-47cb-bb1a-59ef08672a81 None admin] RBAC: Adding query filter params (name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:08.639208 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d89b57-136f-47cb-bb1a-59ef08672a81 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:08.641038 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d89b57-136f-47cb-bb1a-59ef08672a81 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:08.656338 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 283/584] 10.222.0.99 () {56 vars in 1103 bytes} [Thu Jul 14 15:23:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:08.656338 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d89b57-136f-47cb-bb1a-59ef08672a81 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:08.745254 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:08.788179 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:08.814940 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c9a43a-8daf-4486-bd14-318c6a01a9bf None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:08.829749 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d89b57-136f-47cb-bb1a-59ef08672a81 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:08.867052 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d479c4-272b-4fbd-b601-76325fd6948f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:08.868193 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d479c4-272b-4fbd-b601-76325fd6948f None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:08.868490 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d479c4-272b-4fbd-b601-76325fd6948f None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:08.869247 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d479c4-272b-4fbd-b601-76325fd6948f None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:08.874949 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c343861-996d-490e-a6a8-f49f14908d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:08.875983 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c343861-996d-490e-a6a8-f49f14908d46 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:08.876448 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c343861-996d-490e-a6a8-f49f14908d46 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:08.876728 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c343861-996d-490e-a6a8-f49f14908d46 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:08.877796 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d479c4-272b-4fbd-b601-76325fd6948f None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:08.880346 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d479c4-272b-4fbd-b601-76325fd6948f None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:08.885694 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c343861-996d-490e-a6a8-f49f14908d46 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:08.887226 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c343861-996d-490e-a6a8-f49f14908d46 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:08.898830 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-35d479c4-272b-4fbd-b601-76325fd6948f None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 14 15:23:08.898830 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:08.898830 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:08.898830 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:08.898830 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:08.898830 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:08.898830 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:08.898830 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:08.898830 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:08.898830 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:08.898830 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:08.898830 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:08.898830 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 15:23:08.898830 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 15:23:08.898830 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 15:23:08.898830 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 15:23:08.900073 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:08.900073 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:08.900073 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 15:23:08.900073 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:23:08.900073 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 15:23:08.900073 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:23:08.900073 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:08.900073 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:08.900073 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:08.900073 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:08.900073 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:08.900073 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:08.900073 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:08.900073 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:08.900073 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:08.900073 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:08.900073 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:08.900073 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:08.901247 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:08.901247 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:08.901247 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 15:23:08.901247 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 15:23:08.901247 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 15:23:08.901247 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 15:23:08.901247 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 15:23:08.901247 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 15:23:08.901247 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 14 15:23:08.901247 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:23:08.908330 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-6c343861-996d-490e-a6a8-f49f14908d46 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 14 15:23:08.908330 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:08.908330 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:08.908330 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:08.908330 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:08.908330 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:08.908330 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:08.908330 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:08.908330 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:08.908330 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:08.908330 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:08.908330 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:08.908330 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 15:23:08.908330 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 15:23:08.908330 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 15:23:08.908330 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 15:23:08.909473 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:08.909473 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:08.909473 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 15:23:08.909473 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:23:08.909473 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 15:23:08.909473 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:23:08.909473 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:08.909473 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:08.909473 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:08.909473 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:08.909473 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:08.909473 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:08.909473 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:08.909473 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:08.909473 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:08.909473 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:08.909473 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:08.909473 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:08.910784 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:08.910784 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:08.910784 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 15:23:08.910784 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 15:23:08.910784 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 15:23:08.910784 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 15:23:08.910784 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 15:23:08.910784 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 15:23:08.910784 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 14 15:23:08.910784 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:23:08.910784 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 284/586] 10.222.0.99 () {56 vars in 1093 bytes} [Thu Jul 14 15:23:08 2022] GET /identity/v3/users/nova => generated 82 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:08.910784 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c343861-996d-490e-a6a8-f49f14908d46 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:08.911643 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 302/586] 10.222.0.99 () {56 vars in 1102 bytes} [Thu Jul 14 15:23:08 2022] GET /identity/v3/users/neutron => generated 85 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:08.914400 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d479c4-272b-4fbd-b601-76325fd6948f None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:09.041871 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c343861-996d-490e-a6a8-f49f14908d46 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:09.063624 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d479c4-272b-4fbd-b601-76325fd6948f None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:09.090127 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:09.098418 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0b67f6-a307-4244-8ad1-3cf6a93adfda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:09.098626 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbd1b20-b21c-4d2d-9ec7-266e76f02a5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:09.099236 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0b67f6-a307-4244-8ad1-3cf6a93adfda None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:09.099433 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbd1b20-b21c-4d2d-9ec7-266e76f02a5d None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:09.099535 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0b67f6-a307-4244-8ad1-3cf6a93adfda None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:09.099825 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0b67f6-a307-4244-8ad1-3cf6a93adfda None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:09.099989 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbd1b20-b21c-4d2d-9ec7-266e76f02a5d None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:09.100388 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbd1b20-b21c-4d2d-9ec7-266e76f02a5d None admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:09.101340 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbd1b20-b21c-4d2d-9ec7-266e76f02a5d None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:09.101749 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbd1b20-b21c-4d2d-9ec7-266e76f02a5d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:09.104291 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbd1b20-b21c-4d2d-9ec7-266e76f02a5d None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:09.112486 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:09.115313 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0b67f6-a307-4244-8ad1-3cf6a93adfda None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:09.117024 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0b67f6-a307-4244-8ad1-3cf6a93adfda None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:09.130021 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 303/587] 10.222.0.99 () {56 vars in 1139 bytes} [Thu Jul 14 15:23:08 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:09.131228 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbd1b20-b21c-4d2d-9ec7-266e76f02a5d None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:09.137048 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-ac0b67f6-a307-4244-8ad1-3cf6a93adfda None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:23:09.137048 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:09.137048 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:09.137048 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:09.137048 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:09.137048 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:09.137048 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:09.137048 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:09.137048 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:09.137048 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:09.137048 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:09.137048 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:09.137048 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:23:09.137048 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:23:09.137048 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:23:09.137918 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:23:09.137918 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:09.137918 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:09.137918 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:09.137918 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:09.137918 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:09.137918 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:09.137918 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:09.137918 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:09.137918 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:09.137918 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:09.137918 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:09.137918 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:09.137918 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:09.137918 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:09.137918 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:09.137918 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:09.139350 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:23:09.139350 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:23:09.139350 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:23:09.139350 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:23:09.139350 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 285/588] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:23:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:09.140059 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0b67f6-a307-4244-8ad1-3cf6a93adfda None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:09.271032 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbd1b20-b21c-4d2d-9ec7-266e76f02a5d None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:09.272849 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0b67f6-a307-4244-8ad1-3cf6a93adfda None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:09.310416 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1c2406-8108-4bc2-9199-fc90006fa397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:09.311807 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1c2406-8108-4bc2-9199-fc90006fa397 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:09.312336 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1c2406-8108-4bc2-9199-fc90006fa397 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:09.312772 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0e10dd-c1e1-4070-976c-f2b2882c002a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:09.313045 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1c2406-8108-4bc2-9199-fc90006fa397 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:09.313457 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0e10dd-c1e1-4070-976c-f2b2882c002a None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:09.313749 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0e10dd-c1e1-4070-976c-f2b2882c002a None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:09.314032 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0e10dd-c1e1-4070-976c-f2b2882c002a None admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:09.314896 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0e10dd-c1e1-4070-976c-f2b2882c002a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:09.315213 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0e10dd-c1e1-4070-976c-f2b2882c002a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:09.316699 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0e10dd-c1e1-4070-976c-f2b2882c002a None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:09.326729 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1c2406-8108-4bc2-9199-fc90006fa397 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:09.328599 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1c2406-8108-4bc2-9199-fc90006fa397 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:09.342188 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 304/589] 10.222.0.99 () {56 vars in 1133 bytes} [Thu Jul 14 15:23:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:09.343315 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0e10dd-c1e1-4070-976c-f2b2882c002a None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:09.348615 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-4d1c2406-8108-4bc2-9199-fc90006fa397 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:23:09.348615 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:09.348615 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:09.348615 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:09.348615 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:09.348615 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:09.348615 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:09.348615 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:09.348615 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:09.348615 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:09.348615 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:09.348615 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:09.348615 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:23:09.348615 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:23:09.348615 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:23:09.349599 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:23:09.349599 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:09.349599 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:09.349599 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:09.349599 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:09.349599 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:09.349599 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:09.349599 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:09.349599 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:09.349599 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:09.349599 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:09.349599 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:09.349599 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:09.349599 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:09.349599 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:09.349599 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:09.349599 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:09.350456 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:23:09.350456 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:23:09.350456 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:23:09.350456 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:23:09.352756 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 286/590] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:23:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:09.353911 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1c2406-8108-4bc2-9199-fc90006fa397 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:09.399544 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:09.409698 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:09.462147 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0e10dd-c1e1-4070-976c-f2b2882c002a None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:09.470524 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1c2406-8108-4bc2-9199-fc90006fa397 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:09.504929 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06035648-718d-4f98-a615-d489c5000c7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:09.505869 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06035648-718d-4f98-a615-d489c5000c7e None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:09.506164 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06035648-718d-4f98-a615-d489c5000c7e None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:09.506447 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06035648-718d-4f98-a615-d489c5000c7e None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:09.507195 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06035648-718d-4f98-a615-d489c5000c7e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:09.507507 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06035648-718d-4f98-a615-d489c5000c7e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:09.509211 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06035648-718d-4f98-a615-d489c5000c7e None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:09.510046 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec1e33e-b7d4-4c1a-a08a-fff5346c82a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:09.511334 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec1e33e-b7d4-4c1a-a08a-fff5346c82a3 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:09.511889 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec1e33e-b7d4-4c1a-a08a-fff5346c82a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:09.512375 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec1e33e-b7d4-4c1a-a08a-fff5346c82a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:09.523011 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 305/591] 10.222.0.99 () {56 vars in 1109 bytes} [Thu Jul 14 15:23:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:09.523835 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06035648-718d-4f98-a615-d489c5000c7e None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:09.531893 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec1e33e-b7d4-4c1a-a08a-fff5346c82a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:09.533780 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec1e33e-b7d4-4c1a-a08a-fff5346c82a3 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:09.553124 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-7ec1e33e-b7d4-4c1a-a08a-fff5346c82a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:23:09.553124 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:09.553124 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:09.553124 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:09.553124 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:09.553124 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:09.553124 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:09.553124 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:09.553124 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:09.553124 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:09.553124 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:09.553124 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:09.553124 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:23:09.553124 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:23:09.553124 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:23:09.554126 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:23:09.554126 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:09.554126 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:09.554126 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:09.554126 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:09.554126 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:09.554126 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:09.554126 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:09.554126 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:09.554126 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:09.554126 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:09.554126 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:09.554126 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:09.554126 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:09.554126 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:09.554126 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:09.554126 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:09.555024 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:23:09.555024 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:23:09.555024 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:23:09.555024 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:23:09.557310 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 287/592] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:23:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:09.558334 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec1e33e-b7d4-4c1a-a08a-fff5346c82a3 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:09.648291 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06035648-718d-4f98-a615-d489c5000c7e None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:09.657005 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:09.662467 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:09.669184 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec1e33e-b7d4-4c1a-a08a-fff5346c82a3 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:09.697703 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c58e45-8926-4604-864d-092e1d459fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:09.698393 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c58e45-8926-4604-864d-092e1d459fab None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:09.698846 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c58e45-8926-4604-864d-092e1d459fab None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:09.699081 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c58e45-8926-4604-864d-092e1d459fab None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:09.699862 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c58e45-8926-4604-864d-092e1d459fab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:09.700178 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c58e45-8926-4604-864d-092e1d459fab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:09.702222 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c58e45-8926-4604-864d-092e1d459fab None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:09.711196 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a7ab6c-a7b8-423d-9bad-f771d0c23dae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:09.712425 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a7ab6c-a7b8-423d-9bad-f771d0c23dae None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:09.713018 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a7ab6c-a7b8-423d-9bad-f771d0c23dae None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:09.713497 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a7ab6c-a7b8-423d-9bad-f771d0c23dae None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:09.716312 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 306/593] 10.222.0.99 () {56 vars in 1109 bytes} [Thu Jul 14 15:23:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:09.717959 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c58e45-8926-4604-864d-092e1d459fab None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:09.723290 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a7ab6c-a7b8-423d-9bad-f771d0c23dae None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:09.725152 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a7ab6c-a7b8-423d-9bad-f771d0c23dae None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:09.741023 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-08a7ab6c-a7b8-423d-9bad-f771d0c23dae None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 14 15:23:09.741023 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:09.741023 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:09.741023 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:09.741023 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:09.741023 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:09.741023 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:09.741023 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:09.741023 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:09.741023 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:09.741023 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:09.741023 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:09.741023 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 15:23:09.741023 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 15:23:09.741023 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 15:23:09.741023 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 15:23:09.742016 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:09.742016 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:09.742016 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 15:23:09.742016 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:23:09.742016 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 15:23:09.742016 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:23:09.742016 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:09.742016 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:09.742016 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:09.742016 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:09.742016 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:09.742016 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:09.742016 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:09.742016 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:09.742016 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:09.742016 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:09.742016 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:09.742016 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:09.742945 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:09.742945 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:09.742945 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 15:23:09.742945 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 15:23:09.742945 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 15:23:09.742945 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 15:23:09.742945 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 15:23:09.742945 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 15:23:09.742945 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 14 15:23:09.742945 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:23:09.745555 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 288/594] 10.222.0.99 () {56 vars in 1099 bytes} [Thu Jul 14 15:23:09 2022] GET /identity/v3/users/cinder => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:09.746597 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a7ab6c-a7b8-423d-9bad-f771d0c23dae None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:09.831157 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c58e45-8926-4604-864d-092e1d459fab None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:09.854165 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a7ab6c-a7b8-423d-9bad-f771d0c23dae None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:09.874352 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba1315b-4844-470e-ad02-9e23c1a38067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:09.875075 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba1315b-4844-470e-ad02-9e23c1a38067 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:09.875426 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba1315b-4844-470e-ad02-9e23c1a38067 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:09.875922 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba1315b-4844-470e-ad02-9e23c1a38067 None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:09.876777 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba1315b-4844-470e-ad02-9e23c1a38067 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:09.877095 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba1315b-4844-470e-ad02-9e23c1a38067 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:09.878495 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba1315b-4844-470e-ad02-9e23c1a38067 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:09.894777 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a99760e-91ff-4656-b5c9-d45f71f64e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:09.895980 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a99760e-91ff-4656-b5c9-d45f71f64e72 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:09.896471 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 307/595] 10.222.0.99 () {56 vars in 1109 bytes} [Thu Jul 14 15:23:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:09.896684 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a99760e-91ff-4656-b5c9-d45f71f64e72 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:09.897163 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a99760e-91ff-4656-b5c9-d45f71f64e72 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:09.897367 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba1315b-4844-470e-ad02-9e23c1a38067 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:09.905802 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a99760e-91ff-4656-b5c9-d45f71f64e72 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:09.907587 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a99760e-91ff-4656-b5c9-d45f71f64e72 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:09.923255 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-9a99760e-91ff-4656-b5c9-d45f71f64e72 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:23:09.923255 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:09.923255 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:09.923255 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:09.923255 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:09.923255 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:09.923255 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:09.923255 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:09.923255 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:09.923255 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:09.923255 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:09.923255 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:09.923255 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 15:23:09.923255 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 15:23:09.923255 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 15:23:09.924400 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 15:23:09.924400 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:09.924400 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:09.924400 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:09.924400 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:09.924400 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:09.924400 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:09.924400 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:09.924400 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:09.924400 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:09.924400 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:09.924400 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:09.924400 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:09.924400 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:09.924400 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:09.924400 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:09.924400 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:09.925284 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 15:23:09.925284 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 15:23:09.925284 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 15:23:09.925284 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 15:23:09.925284 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 15:23:09.925284 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 15:23:09.925284 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:23:09.925284 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:23:09.927714 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 289/596] 10.222.0.99 () {56 vars in 1111 bytes} [Thu Jul 14 15:23:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:09.928738 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a99760e-91ff-4656-b5c9-d45f71f64e72 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:09.928933 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:09.947909 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:10.022157 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba1315b-4844-470e-ad02-9e23c1a38067 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:10.036891 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a99760e-91ff-4656-b5c9-d45f71f64e72 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:10.063221 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c06e55-d98e-4a38-911c-93ab03973930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:10.064019 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c06e55-d98e-4a38-911c-93ab03973930 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:10.064246 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c06e55-d98e-4a38-911c-93ab03973930 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:10.064555 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c06e55-d98e-4a38-911c-93ab03973930 None admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:10.065394 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c06e55-d98e-4a38-911c-93ab03973930 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:10.065711 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c06e55-d98e-4a38-911c-93ab03973930 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:10.067409 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c06e55-d98e-4a38-911c-93ab03973930 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:10.075320 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127108de-7fa2-4c87-8c98-15231de75f3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:10.076974 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127108de-7fa2-4c87-8c98-15231de75f3b None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:10.077677 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127108de-7fa2-4c87-8c98-15231de75f3b None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:10.078233 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127108de-7fa2-4c87-8c98-15231de75f3b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:10.085816 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127108de-7fa2-4c87-8c98-15231de75f3b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:10.087504 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127108de-7fa2-4c87-8c98-15231de75f3b None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:10.087856 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 308/597] 10.222.0.99 () {56 vars in 1137 bytes} [Thu Jul 14 15:23:09 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:10.088804 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c06e55-d98e-4a38-911c-93ab03973930 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:10.103058 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-127108de-7fa2-4c87-8c98-15231de75f3b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:23:10.103058 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:10.103058 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:10.103058 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:10.103058 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:10.103058 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:10.103058 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:10.103058 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:10.103058 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:10.103058 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:10.103058 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:10.103058 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:10.103058 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 15:23:10.103058 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 15:23:10.103058 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 15:23:10.104103 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 15:23:10.104103 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:10.104103 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:10.104103 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:10.104103 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:10.104103 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:10.104103 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:10.104103 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:10.104103 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:10.104103 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:10.104103 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:10.104103 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:10.104103 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:10.104103 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:10.104103 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:10.104103 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:10.104103 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:10.104987 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 15:23:10.104987 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 15:23:10.104987 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 15:23:10.104987 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 15:23:10.104987 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 15:23:10.104987 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 15:23:10.104987 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:23:10.104987 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:23:10.107490 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 290/598] 10.222.0.99 () {56 vars in 1111 bytes} [Thu Jul 14 15:23:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:10.108618 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127108de-7fa2-4c87-8c98-15231de75f3b None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:10.189812 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:10.191435 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:10.212584 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c06e55-d98e-4a38-911c-93ab03973930 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:10.217163 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127108de-7fa2-4c87-8c98-15231de75f3b None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:10.254351 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3638a374-aa0f-4fd7-9099-65ed6506fc72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:10.255412 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887b68f7-f220-457d-b30e-f49f00c50e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:10.255854 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3638a374-aa0f-4fd7-9099-65ed6506fc72 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:10.256129 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887b68f7-f220-457d-b30e-f49f00c50e01 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:10.256414 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887b68f7-f220-457d-b30e-f49f00c50e01 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:10.256693 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887b68f7-f220-457d-b30e-f49f00c50e01 None admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:10.257009 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3638a374-aa0f-4fd7-9099-65ed6506fc72 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:10.257492 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887b68f7-f220-457d-b30e-f49f00c50e01 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:10.257606 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3638a374-aa0f-4fd7-9099-65ed6506fc72 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:10.257798 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887b68f7-f220-457d-b30e-f49f00c50e01 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:10.260003 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887b68f7-f220-457d-b30e-f49f00c50e01 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:10.274011 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3638a374-aa0f-4fd7-9099-65ed6506fc72 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:10.275934 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3638a374-aa0f-4fd7-9099-65ed6506fc72 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:10.277810 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 309/599] 10.222.0.99 () {56 vars in 1148 bytes} [Thu Jul 14 15:23:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:10.278592 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887b68f7-f220-457d-b30e-f49f00c50e01 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:10.299548 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-3638a374-aa0f-4fd7-9099-65ed6506fc72 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:23:10.299548 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:10.299548 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:10.299548 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:10.299548 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:10.299548 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:10.299548 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:10.299548 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:10.299548 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:10.299548 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:10.299548 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:10.299548 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:10.299548 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:23:10.299548 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:23:10.299548 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:23:10.300514 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:23:10.300514 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:10.300514 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:10.300514 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:10.300514 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:10.300514 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:10.300514 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:10.300514 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:10.300514 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:10.300514 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:10.300514 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:10.300514 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:10.300514 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:10.300514 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:10.300514 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:10.300514 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:10.300514 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:10.301390 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:23:10.301390 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:23:10.301390 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:23:10.301390 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:23:10.303789 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 291/600] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:23:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:10.304803 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3638a374-aa0f-4fd7-9099-65ed6506fc72 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:10.418161 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3638a374-aa0f-4fd7-9099-65ed6506fc72 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:10.419111 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887b68f7-f220-457d-b30e-f49f00c50e01 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:10.447653 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:10.457222 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567c74b9-aed7-4224-bc6e-7ce034f4e7b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:10.458140 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567c74b9-aed7-4224-bc6e-7ce034f4e7b4 None admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:10.458437 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567c74b9-aed7-4224-bc6e-7ce034f4e7b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:10.458710 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567c74b9-aed7-4224-bc6e-7ce034f4e7b4 None admin] PATH_INFO: `/v3/projects/9048d1a6f00443ec92f11471babe5a56/users/0d0261759f9743b1a13a760ca9159305/roles/74e3b597938f49d095cb424c54b5a811` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:10.468334 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2093aa-ca01-4e69-84e6-c0ab5fa85b90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:10.469763 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2093aa-ca01-4e69-84e6-c0ab5fa85b90 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:10.470238 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2093aa-ca01-4e69-84e6-c0ab5fa85b90 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:10.470673 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2093aa-ca01-4e69-84e6-c0ab5fa85b90 None admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:10.471667 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2093aa-ca01-4e69-84e6-c0ab5fa85b90 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:10.472132 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2093aa-ca01-4e69-84e6-c0ab5fa85b90 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:10.474100 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2093aa-ca01-4e69-84e6-c0ab5fa85b90 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:10.476957 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567c74b9-aed7-4224-bc6e-7ce034f4e7b4 None admin] RBAC: Authorizing `identity:create_grant(project_id=9048d1a6f00443ec92f11471babe5a56, user_id=0d0261759f9743b1a13a760ca9159305, role_id=74e3b597938f49d095cb424c54b5a811)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:10.478404 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567c74b9-aed7-4224-bc6e-7ce034f4e7b4 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:10.490024 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 310/601] 10.222.0.99 () {56 vars in 1148 bytes} [Thu Jul 14 15:23:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:10.491440 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2093aa-ca01-4e69-84e6-c0ab5fa85b90 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:10.494126 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 292/602] 10.222.0.99 () {58 vars in 1439 bytes} [Thu Jul 14 15:23:10 2022] PUT /identity/v3/projects/9048d1a6f00443ec92f11471babe5a56/users/0d0261759f9743b1a13a760ca9159305/roles/74e3b597938f49d095cb424c54b5a811 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:23:10.496379 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:10.499807 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567c74b9-aed7-4224-bc6e-7ce034f4e7b4 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:10.660117 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567c74b9-aed7-4224-bc6e-7ce034f4e7b4 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:10.702385 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2093aa-ca01-4e69-84e6-c0ab5fa85b90 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:10.728386 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed904b8-e3f4-4b55-888b-0aa9cea13d70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:10.729918 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed904b8-e3f4-4b55-888b-0aa9cea13d70 None admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:10.730901 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed904b8-e3f4-4b55-888b-0aa9cea13d70 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:10.731326 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed904b8-e3f4-4b55-888b-0aa9cea13d70 None admin] PATH_INFO: `/v3/projects/9048d1a6f00443ec92f11471babe5a56/users/3a9b1c2622934c5185f3b5f1ea623bc0/roles/74e3b597938f49d095cb424c54b5a811` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:10.747133 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1da58f-89be-4dff-9894-926631921543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:10.747933 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1da58f-89be-4dff-9894-926631921543 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:10.748260 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1da58f-89be-4dff-9894-926631921543 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:10.748610 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1da58f-89be-4dff-9894-926631921543 None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:10.749668 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1da58f-89be-4dff-9894-926631921543 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:10.750220 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1da58f-89be-4dff-9894-926631921543 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:10.751776 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1da58f-89be-4dff-9894-926631921543 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:10.753264 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed904b8-e3f4-4b55-888b-0aa9cea13d70 None admin] RBAC: Authorizing `identity:create_grant(project_id=9048d1a6f00443ec92f11471babe5a56, user_id=3a9b1c2622934c5185f3b5f1ea623bc0, role_id=74e3b597938f49d095cb424c54b5a811)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:10.755838 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed904b8-e3f4-4b55-888b-0aa9cea13d70 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:10.771037 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 311/603] 10.222.0.99 () {56 vars in 1109 bytes} [Thu Jul 14 15:23:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:10.774768 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 293/604] 10.222.0.99 () {58 vars in 1439 bytes} [Thu Jul 14 15:23:10 2022] PUT /identity/v3/projects/9048d1a6f00443ec92f11471babe5a56/users/3a9b1c2622934c5185f3b5f1ea623bc0/roles/74e3b597938f49d095cb424c54b5a811 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:23:10.781528 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1da58f-89be-4dff-9894-926631921543 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:10.794488 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362de5d5-0102-46cb-be42-a4d43016664e None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:10.794812 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362de5d5-0102-46cb-be42-a4d43016664e None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:10.795722 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362de5d5-0102-46cb-be42-a4d43016664e None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:10.801903 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 294/605] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:23:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 15:23:10.804261 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6801c3f1-1dfa-40a3-91e2-d74ff141185a None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:10.804261 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6801c3f1-1dfa-40a3-91e2-d74ff141185a None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:10.804536 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6801c3f1-1dfa-40a3-91e2-d74ff141185a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:10.823730 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-6801c3f1-1dfa-40a3-91e2-d74ff141185a None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:23:10.825387 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-6801c3f1-1dfa-40a3-91e2-d74ff141185a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:23:10.855792 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:10.875358 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:10.940126 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1da58f-89be-4dff-9894-926631921543 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:10.942652 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 295/606] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:23:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 15:23:10.944739 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e269e36-2d60-4812-b1ee-462ac40893be None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:10.944951 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e269e36-2d60-4812-b1ee-462ac40893be None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:10.945233 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e269e36-2d60-4812-b1ee-462ac40893be None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:10.952172 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 296/607] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:23:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:23:10.953082 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20c8cc9-8493-416f-8d69-ab94baec3328 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:10.953323 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20c8cc9-8493-416f-8d69-ab94baec3328 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:10.953611 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20c8cc9-8493-416f-8d69-ab94baec3328 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:10.974856 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-b20c8cc9-8493-416f-8d69-ab94baec3328 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:23:10.976131 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b20c8cc9-8493-416f-8d69-ab94baec3328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:23:10.999635 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3316a7-891e-4318-bcb5-db1bf5123118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:11.003620 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3316a7-891e-4318-bcb5-db1bf5123118 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:11.003620 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3316a7-891e-4318-bcb5-db1bf5123118 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:11.003620 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3316a7-891e-4318-bcb5-db1bf5123118 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:11.013122 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3316a7-891e-4318-bcb5-db1bf5123118 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:11.016062 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3316a7-891e-4318-bcb5-db1bf5123118 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:11.033077 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-9c3316a7-891e-4318-bcb5-db1bf5123118 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:23:11.033077 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:11.033077 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:11.033077 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:11.033077 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:11.033077 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:11.033077 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:11.033077 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:11.033077 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:11.033077 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:11.033077 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:11.033077 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:11.033077 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 15:23:11.033077 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 15:23:11.033077 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 15:23:11.034069 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 15:23:11.034069 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:11.034069 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:11.034069 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:11.034069 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:11.034069 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:11.034069 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:11.034069 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:11.034069 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:11.034069 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:11.034069 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:11.034069 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:11.034069 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:11.034069 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:11.034069 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:11.034069 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:11.034069 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:11.035819 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 15:23:11.035819 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 15:23:11.035819 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 15:23:11.035819 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 15:23:11.035819 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 15:23:11.035819 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 15:23:11.035819 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:23:11.035819 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:23:11.035819 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 312/608] 10.222.0.99 () {56 vars in 1111 bytes} [Thu Jul 14 15:23:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:11.035819 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3316a7-891e-4318-bcb5-db1bf5123118 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:11.075130 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 297/609] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:23:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:23:11.076044 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20c8cc9-8493-416f-8d69-ab94baec3328 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:11.166755 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3316a7-891e-4318-bcb5-db1bf5123118 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:11.195182 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:11.206827 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:11.211922 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df97c13-5d60-4424-a285-d234f0b18333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:11.212977 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df97c13-5d60-4424-a285-d234f0b18333 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:11.213366 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df97c13-5d60-4424-a285-d234f0b18333 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:11.213729 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df97c13-5d60-4424-a285-d234f0b18333 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:11.224242 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20c8cc9-8493-416f-8d69-ab94baec3328 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:11.226452 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df97c13-5d60-4424-a285-d234f0b18333 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:23:11.235326 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-8df97c13-5d60-4424-a285-d234f0b18333 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 15:23:11.235326 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:11.235326 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:11.235326 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:11.235326 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:11.235326 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:11.235326 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:11.235326 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:11.235326 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:11.235326 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:11.235326 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:11.235326 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:11.235326 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:23:11.235326 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:23:11.235326 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:23:11.235326 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:23:11.237662 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:23:11.237662 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:23:11.237662 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:23:11.237662 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:23:11.237662 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:11.237662 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:11.237662 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:11.237662 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:11.237662 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:11.237662 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:11.237662 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:11.237662 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:11.237662 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:11.237662 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:11.237662 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:11.237662 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:11.240039 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:11.240039 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:11.240039 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:11.240039 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:11.240039 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:23:11.240039 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:23:11.240039 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:23:11.240039 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:23:11.240039 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:23:11.240039 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:23:11.240039 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 15:23:11.240039 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:23:11.240039 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 313/610] 10.222.0.99 () {56 vars in 1102 bytes} [Thu Jul 14 15:23:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:11.240039 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df97c13-5d60-4424-a285-d234f0b18333 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:11.280934 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4ad397-3e6f-4aa3-be25-cf5904e081ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:11.281375 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4ad397-3e6f-4aa3-be25-cf5904e081ed None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:11.281660 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4ad397-3e6f-4aa3-be25-cf5904e081ed None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:11.281939 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4ad397-3e6f-4aa3-be25-cf5904e081ed None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:11.282740 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4ad397-3e6f-4aa3-be25-cf5904e081ed None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:11.283052 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4ad397-3e6f-4aa3-be25-cf5904e081ed None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:11.285541 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4ad397-3e6f-4aa3-be25-cf5904e081ed None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:11.301806 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 298/611] 10.222.0.99 () {56 vars in 1148 bytes} [Thu Jul 14 15:23:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:11.302770 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4ad397-3e6f-4aa3-be25-cf5904e081ed None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:11.377881 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df97c13-5d60-4424-a285-d234f0b18333 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:11.421861 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bb2198-80e6-4e62-9eea-e08237befc50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:11.422829 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bb2198-80e6-4e62-9eea-e08237befc50 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:11.423218 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bb2198-80e6-4e62-9eea-e08237befc50 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:11.423879 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bb2198-80e6-4e62-9eea-e08237befc50 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:11.431096 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4ad397-3e6f-4aa3-be25-cf5904e081ed None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:11.436791 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bb2198-80e6-4e62-9eea-e08237befc50 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:23:11.444745 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-58bb2198-80e6-4e62-9eea-e08237befc50 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 15:23:11.444745 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:11.444745 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:11.444745 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:11.444745 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:11.444745 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:11.444745 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:11.444745 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:11.444745 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:11.444745 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:11.444745 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:11.444745 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:11.444745 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:23:11.444745 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:23:11.444745 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:23:11.444745 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:23:11.445887 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:23:11.445887 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:23:11.445887 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:23:11.445887 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:23:11.445887 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:11.445887 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:11.445887 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:11.445887 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:11.445887 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:11.445887 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:11.445887 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:11.445887 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:11.445887 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:11.445887 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:11.445887 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:11.445887 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:11.446942 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:11.446942 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:11.446942 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:11.446942 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:11.446942 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:23:11.446942 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:23:11.446942 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:23:11.446942 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:23:11.446942 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:23:11.446942 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:23:11.446942 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 15:23:11.446942 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:23:11.448972 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 314/612] 10.222.0.99 () {56 vars in 1102 bytes} [Thu Jul 14 15:23:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:11.449923 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bb2198-80e6-4e62-9eea-e08237befc50 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:11.480425 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151d69dc-3f54-46f0-ad5e-a796fc8df194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:11.481618 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151d69dc-3f54-46f0-ad5e-a796fc8df194 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:11.482061 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151d69dc-3f54-46f0-ad5e-a796fc8df194 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:11.483484 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151d69dc-3f54-46f0-ad5e-a796fc8df194 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:11.484441 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151d69dc-3f54-46f0-ad5e-a796fc8df194 None admin] RBAC: Adding query filter params (name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:11.485182 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151d69dc-3f54-46f0-ad5e-a796fc8df194 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:11.487167 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151d69dc-3f54-46f0-ad5e-a796fc8df194 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:11.499147 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 299/613] 10.222.0.99 () {56 vars in 1103 bytes} [Thu Jul 14 15:23:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:11.500248 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151d69dc-3f54-46f0-ad5e-a796fc8df194 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:11.514861 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:11.523810 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:11.586451 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bb2198-80e6-4e62-9eea-e08237befc50 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:11.628587 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151d69dc-3f54-46f0-ad5e-a796fc8df194 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:11.632279 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb0b818-bcad-488d-85de-d72024565df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:11.634431 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb0b818-bcad-488d-85de-d72024565df6 None admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:11.634881 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb0b818-bcad-488d-85de-d72024565df6 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:11.635338 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb0b818-bcad-488d-85de-d72024565df6 None admin] PATH_INFO: `/v3/projects/9048d1a6f00443ec92f11471babe5a56/users/a3e1a2b83f3d4d88ba5084dfaffd22f3/roles/74e3b597938f49d095cb424c54b5a811` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:11.654946 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb0b818-bcad-488d-85de-d72024565df6 None admin] RBAC: Authorizing `identity:create_grant(project_id=9048d1a6f00443ec92f11471babe5a56, user_id=a3e1a2b83f3d4d88ba5084dfaffd22f3, role_id=74e3b597938f49d095cb424c54b5a811)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:11.657166 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb0b818-bcad-488d-85de-d72024565df6 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:11.674566 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dd52b2-3fec-400b-af8e-d46ffd108106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:11.675429 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 315/614] 10.222.0.99 () {58 vars in 1439 bytes} [Thu Jul 14 15:23:11 2022] PUT /identity/v3/projects/9048d1a6f00443ec92f11471babe5a56/users/a3e1a2b83f3d4d88ba5084dfaffd22f3/roles/74e3b597938f49d095cb424c54b5a811 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:23:11.675572 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dd52b2-3fec-400b-af8e-d46ffd108106 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:11.676076 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dd52b2-3fec-400b-af8e-d46ffd108106 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:11.676547 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dd52b2-3fec-400b-af8e-d46ffd108106 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:11.676910 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb0b818-bcad-488d-85de-d72024565df6 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:11.677919 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dd52b2-3fec-400b-af8e-d46ffd108106 None admin] RBAC: Adding query filter params (name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:11.678344 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dd52b2-3fec-400b-af8e-d46ffd108106 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:11.679961 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dd52b2-3fec-400b-af8e-d46ffd108106 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:11.692958 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 300/615] 10.222.0.99 () {56 vars in 1103 bytes} [Thu Jul 14 15:23:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:11.698564 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dd52b2-3fec-400b-af8e-d46ffd108106 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:11.816322 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:11.825637 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb0b818-bcad-488d-85de-d72024565df6 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:11.857231 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:11.897538 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fae15a5-7a14-418e-a8e3-237925477380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:11.898299 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fae15a5-7a14-418e-a8e3-237925477380 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:11.898696 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fae15a5-7a14-418e-a8e3-237925477380 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:11.898861 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fae15a5-7a14-418e-a8e3-237925477380 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:11.912200 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fae15a5-7a14-418e-a8e3-237925477380 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:11.913745 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fae15a5-7a14-418e-a8e3-237925477380 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:11.916274 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dd52b2-3fec-400b-af8e-d46ffd108106 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:11.933907 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-2fae15a5-7a14-418e-a8e3-237925477380 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:23:11.933907 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:11.933907 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:11.933907 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:11.933907 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:11.933907 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:11.933907 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:11.933907 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:11.933907 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:11.933907 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:11.933907 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:11.933907 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:11.933907 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:23:11.933907 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:23:11.933907 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:23:11.935046 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:23:11.935046 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:11.935046 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:11.935046 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:11.935046 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:11.935046 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:11.935046 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:11.935046 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:11.935046 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:11.935046 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:11.935046 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:11.935046 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:11.935046 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:11.935046 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:11.935046 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:11.935046 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:11.935046 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:11.936604 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:23:11.936604 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:23:11.936604 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:23:11.936604 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:23:11.936604 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 316/616] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:23:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:11.941368 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fae15a5-7a14-418e-a8e3-237925477380 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:11.966953 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d47364-030a-4045-a5d4-23cef34a5e42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:11.969224 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d47364-030a-4045-a5d4-23cef34a5e42 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:11.969515 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d47364-030a-4045-a5d4-23cef34a5e42 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:11.969796 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d47364-030a-4045-a5d4-23cef34a5e42 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:11.987830 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d47364-030a-4045-a5d4-23cef34a5e42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:11.989597 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d47364-030a-4045-a5d4-23cef34a5e42 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:12.015036 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-68d47364-030a-4045-a5d4-23cef34a5e42 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:23:12.015036 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:12.015036 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:12.015036 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:12.015036 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:12.015036 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:12.015036 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:12.015036 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:12.015036 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:12.015036 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:12.015036 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:12.015036 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:12.015036 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:23:12.015036 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:23:12.015036 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:23:12.016033 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:23:12.016033 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:12.016033 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:12.016033 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:12.016033 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:12.016033 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:12.016033 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:12.016033 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:12.016033 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:12.016033 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:12.016033 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:12.016033 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:12.016033 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:12.016033 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:12.016033 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:12.016033 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:12.016033 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:12.016904 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:23:12.016904 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:23:12.016904 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:23:12.016904 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:23:12.020464 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 301/617] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:23:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:12.031779 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d47364-030a-4045-a5d4-23cef34a5e42 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:12.170983 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fae15a5-7a14-418e-a8e3-237925477380 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:12.218453 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d47364-030a-4045-a5d4-23cef34a5e42 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:12.244645 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0777d474-2e5e-47ef-ad4b-7890f0e7cfa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:12.245302 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0777d474-2e5e-47ef-ad4b-7890f0e7cfa9 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:12.245590 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0777d474-2e5e-47ef-ad4b-7890f0e7cfa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:12.245869 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0777d474-2e5e-47ef-ad4b-7890f0e7cfa9 None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:12.246604 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0777d474-2e5e-47ef-ad4b-7890f0e7cfa9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:12.246911 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0777d474-2e5e-47ef-ad4b-7890f0e7cfa9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:12.248896 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0777d474-2e5e-47ef-ad4b-7890f0e7cfa9 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:12.267400 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:12.272216 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 317/618] 10.222.0.99 () {56 vars in 1109 bytes} [Thu Jul 14 15:23:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:23:12.277932 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0777d474-2e5e-47ef-ad4b-7890f0e7cfa9 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:12.290961 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6f234d-cf47-4814-a3ae-7d693754bbda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:12.291658 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6f234d-cf47-4814-a3ae-7d693754bbda None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:12.291950 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6f234d-cf47-4814-a3ae-7d693754bbda None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:12.292231 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6f234d-cf47-4814-a3ae-7d693754bbda None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:12.292967 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6f234d-cf47-4814-a3ae-7d693754bbda None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:12.293282 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6f234d-cf47-4814-a3ae-7d693754bbda None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:12.294659 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6f234d-cf47-4814-a3ae-7d693754bbda None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:12.299956 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:12.316405 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 302/619] 10.222.0.99 () {56 vars in 1109 bytes} [Thu Jul 14 15:23:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:12.322345 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6f234d-cf47-4814-a3ae-7d693754bbda None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:12.422427 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0777d474-2e5e-47ef-ad4b-7890f0e7cfa9 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:12.494093 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1880bc-1207-45b9-b2ed-9b9d1276a382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:12.495153 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1880bc-1207-45b9-b2ed-9b9d1276a382 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:12.495621 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1880bc-1207-45b9-b2ed-9b9d1276a382 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:12.496145 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1880bc-1207-45b9-b2ed-9b9d1276a382 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:12.513172 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1880bc-1207-45b9-b2ed-9b9d1276a382 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:12.516295 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1880bc-1207-45b9-b2ed-9b9d1276a382 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:12.551480 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-2d1880bc-1207-45b9-b2ed-9b9d1276a382 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 14 15:23:12.551480 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:12.551480 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:12.551480 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:12.551480 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:12.551480 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:12.551480 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:12.551480 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:12.551480 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:12.551480 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:12.551480 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:12.551480 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:12.551480 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 15:23:12.551480 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 15:23:12.551480 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 15:23:12.551480 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 15:23:12.552835 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:12.552835 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:12.552835 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 15:23:12.552835 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:23:12.552835 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 15:23:12.552835 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:23:12.552835 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:12.552835 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:12.552835 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:12.552835 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:12.552835 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:12.552835 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:12.552835 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:12.552835 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:12.552835 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:12.552835 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:12.552835 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:12.552835 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:12.554024 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:12.554024 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:12.554024 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 15:23:12.554024 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 15:23:12.554024 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 15:23:12.554024 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 15:23:12.554024 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 15:23:12.554024 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 15:23:12.554024 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 14 15:23:12.554024 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:23:12.556512 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 318/620] 10.222.0.99 () {56 vars in 1096 bytes} [Thu Jul 14 15:23:12 2022] GET /identity/v3/users/swift => generated 83 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 15:23:12.563742 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1880bc-1207-45b9-b2ed-9b9d1276a382 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:12.582672 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6f234d-cf47-4814-a3ae-7d693754bbda None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:12.644875 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:12.680479 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9bd39a-e388-4bf8-8997-ba56a2fba13d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:12.681226 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9bd39a-e388-4bf8-8997-ba56a2fba13d None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:12.681515 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9bd39a-e388-4bf8-8997-ba56a2fba13d None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:12.681796 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9bd39a-e388-4bf8-8997-ba56a2fba13d None admin] PATH_INFO: `/v3/users/glance` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:12.698127 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1880bc-1207-45b9-b2ed-9b9d1276a382 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:12.704865 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9bd39a-e388-4bf8-8997-ba56a2fba13d None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:12.706364 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9bd39a-e388-4bf8-8997-ba56a2fba13d None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:12.737068 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-bb9bd39a-e388-4bf8-8997-ba56a2fba13d None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 14 15:23:12.737068 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:12.737068 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:12.737068 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:12.737068 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:12.737068 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:12.737068 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:12.737068 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:12.737068 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:12.737068 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:12.737068 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:12.737068 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:12.737068 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 15:23:12.737068 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 15:23:12.737068 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 15:23:12.737068 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 15:23:12.738084 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:12.738084 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:12.738084 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 15:23:12.738084 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:23:12.738084 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 15:23:12.738084 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:23:12.738084 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:12.738084 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:12.738084 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:12.738084 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:12.738084 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:12.738084 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:12.738084 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:12.738084 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:12.738084 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:12.738084 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:12.738084 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:12.738084 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:12.739008 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:12.739008 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:12.739008 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 15:23:12.739008 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 15:23:12.739008 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 15:23:12.739008 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 15:23:12.739008 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 15:23:12.739008 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 15:23:12.739008 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 14 15:23:12.739008 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:23:12.743856 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 303/621] 10.222.0.99 () {56 vars in 1099 bytes} [Thu Jul 14 15:23:12 2022] GET /identity/v3/users/glance => generated 84 bytes in 422 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:12.754859 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9bd39a-e388-4bf8-8997-ba56a2fba13d None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:12.761406 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeee4464-3a11-40de-b7c2-3e8219c04c63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:12.762368 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeee4464-3a11-40de-b7c2-3e8219c04c63 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:12.762805 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeee4464-3a11-40de-b7c2-3e8219c04c63 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:12.763264 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeee4464-3a11-40de-b7c2-3e8219c04c63 None admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:12.764711 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeee4464-3a11-40de-b7c2-3e8219c04c63 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:12.765565 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeee4464-3a11-40de-b7c2-3e8219c04c63 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:12.768240 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeee4464-3a11-40de-b7c2-3e8219c04c63 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:12.814407 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 319/622] 10.222.0.99 () {56 vars in 1135 bytes} [Thu Jul 14 15:23:12 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:12.821756 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeee4464-3a11-40de-b7c2-3e8219c04c63 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:12.858485 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:12.981602 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeee4464-3a11-40de-b7c2-3e8219c04c63 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:12.991230 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9bd39a-e388-4bf8-8997-ba56a2fba13d None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:13.051983 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:13.061538 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86cb56e-464e-46c8-a806-ccbe522e93bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:13.062324 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86cb56e-464e-46c8-a806-ccbe522e93bf None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:13.062734 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86cb56e-464e-46c8-a806-ccbe522e93bf None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:13.063085 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86cb56e-464e-46c8-a806-ccbe522e93bf None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:13.064474 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71cf56d-2af6-47a9-8162-51db05d595c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:13.064652 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86cb56e-464e-46c8-a806-ccbe522e93bf None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:13.064984 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86cb56e-464e-46c8-a806-ccbe522e93bf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:13.066428 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71cf56d-2af6-47a9-8162-51db05d595c6 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:13.066931 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86cb56e-464e-46c8-a806-ccbe522e93bf None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:13.067304 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71cf56d-2af6-47a9-8162-51db05d595c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:13.067837 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71cf56d-2af6-47a9-8162-51db05d595c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:13.083304 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71cf56d-2af6-47a9-8162-51db05d595c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:13.086293 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71cf56d-2af6-47a9-8162-51db05d595c6 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:13.090291 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 304/623] 10.222.0.99 () {56 vars in 1137 bytes} [Thu Jul 14 15:23:12 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:13.092631 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc74cd9-367e-45f8-8c60-3c3e48094570 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:13.093375 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc74cd9-367e-45f8-8c60-3c3e48094570 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:13.093830 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc74cd9-367e-45f8-8c60-3c3e48094570 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:13.099708 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 305/624] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:23:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:23:13.100723 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc74cd9-367e-45f8-8c60-3c3e48094570 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:13.115504 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-a71cf56d-2af6-47a9-8162-51db05d595c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:23:13.115504 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:13.115504 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:13.115504 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:13.115504 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:13.115504 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:13.115504 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:13.115504 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:13.115504 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:13.115504 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:13.115504 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:13.115504 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:13.115504 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:23:13.115504 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:23:13.115504 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:23:13.116631 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:23:13.116631 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:13.116631 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:13.116631 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:13.116631 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:13.116631 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:13.116631 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:13.116631 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:13.116631 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:13.116631 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:13.116631 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:13.116631 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:13.116631 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:13.116631 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:13.116631 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:13.116631 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:13.116631 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:13.117633 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:23:13.117633 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:23:13.117633 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:23:13.117633 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:23:13.121492 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 320/625] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:23:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:13.124029 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ce6170-a6a9-4d46-98d7-a4b39baf5ae9 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:13.124737 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ce6170-a6a9-4d46-98d7-a4b39baf5ae9 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:13.125416 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ce6170-a6a9-4d46-98d7-a4b39baf5ae9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:13.152326 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-a7ce6170-a6a9-4d46-98d7-a4b39baf5ae9 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:23:13.153828 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-a7ce6170-a6a9-4d46-98d7-a4b39baf5ae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:23:13.251665 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc74cd9-367e-45f8-8c60-3c3e48094570 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:13.255983 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:13.257345 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 321/626] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:23:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:23:13.265295 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ce6170-a6a9-4d46-98d7-a4b39baf5ae9 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:13.311406 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935f6e98-c861-4d2d-a3b2-dd3f34218b7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:13.312697 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935f6e98-c861-4d2d-a3b2-dd3f34218b7e None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:13.313253 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935f6e98-c861-4d2d-a3b2-dd3f34218b7e None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:13.313734 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935f6e98-c861-4d2d-a3b2-dd3f34218b7e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:13.400877 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935f6e98-c861-4d2d-a3b2-dd3f34218b7e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:13.402334 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935f6e98-c861-4d2d-a3b2-dd3f34218b7e None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:13.423043 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ce6170-a6a9-4d46-98d7-a4b39baf5ae9 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:13.431552 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:13.468630 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ab7980-b469-4de3-a96a-ec071df65641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:13.469447 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ab7980-b469-4de3-a96a-ec071df65641 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:13.469958 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ab7980-b469-4de3-a96a-ec071df65641 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:13.470433 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ab7980-b469-4de3-a96a-ec071df65641 None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:13.471340 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ab7980-b469-4de3-a96a-ec071df65641 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:13.471680 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ab7980-b469-4de3-a96a-ec071df65641 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:13.473197 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ab7980-b469-4de3-a96a-ec071df65641 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:13.492453 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 322/627] 10.222.0.99 () {56 vars in 1109 bytes} [Thu Jul 14 15:23:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:13.492453 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ab7980-b469-4de3-a96a-ec071df65641 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:13.554150 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 306/628] 10.222.0.99 () {58 vars in 1282 bytes} [Thu Jul 14 15:23:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 15:23:13.556008 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51118a5f-375a-41d7-bc63-2d534be669fe None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:13.556640 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51118a5f-375a-41d7-bc63-2d534be669fe None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:13.557263 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51118a5f-375a-41d7-bc63-2d534be669fe None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:13.559881 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 307/629] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:23:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:23:13.561800 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51118a5f-375a-41d7-bc63-2d534be669fe None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:13.565888 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:13.645446 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ab7980-b469-4de3-a96a-ec071df65641 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:13.684639 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96207371-9010-4a97-8232-8a5a8ca7c4ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:13.685477 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96207371-9010-4a97-8232-8a5a8ca7c4ec None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:13.686421 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96207371-9010-4a97-8232-8a5a8ca7c4ec None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:13.686614 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96207371-9010-4a97-8232-8a5a8ca7c4ec None admin] PATH_INFO: `/v3/roles/service` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:13.698057 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96207371-9010-4a97-8232-8a5a8ca7c4ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:23:13.703770 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51118a5f-375a-41d7-bc63-2d534be669fe None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:13.706547 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-96207371-9010-4a97-8232-8a5a8ca7c4ec None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 15:23:13.706547 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:13.706547 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:13.706547 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:13.706547 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:13.706547 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:13.706547 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:13.706547 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:13.706547 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:13.706547 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:13.706547 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:13.706547 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:13.706547 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:23:13.706547 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:23:13.706547 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:23:13.706547 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:23:13.707697 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:23:13.707697 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:23:13.707697 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:23:13.707697 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:23:13.707697 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:13.707697 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:13.707697 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:13.707697 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:13.707697 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:13.707697 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:13.707697 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:13.707697 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:13.707697 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:13.707697 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:13.707697 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:13.707697 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:13.708531 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:13.708531 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:13.708531 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:13.708531 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:13.708531 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:23:13.708531 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:23:13.708531 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:23:13.708531 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:23:13.708531 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:23:13.708531 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:23:13.708531 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 15:23:13.708531 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:23:13.708531 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 323/630] 10.222.0.99 () {56 vars in 1102 bytes} [Thu Jul 14 15:23:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:13.709237 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96207371-9010-4a97-8232-8a5a8ca7c4ec None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:13.744022 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:13.747977 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec111012-2065-40ec-b755-fa00cadf3295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:13.749703 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec111012-2065-40ec-b755-fa00cadf3295 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:13.749847 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec111012-2065-40ec-b755-fa00cadf3295 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:13.750193 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec111012-2065-40ec-b755-fa00cadf3295 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:13.760058 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec111012-2065-40ec-b755-fa00cadf3295 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:13.761510 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec111012-2065-40ec-b755-fa00cadf3295 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:13.776651 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-ec111012-2065-40ec-b755-fa00cadf3295 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:23:13.776651 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:13.776651 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:13.776651 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:13.776651 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:13.776651 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:13.776651 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:13.776651 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:13.776651 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:13.776651 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:13.776651 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:13.776651 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:13.776651 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 15:23:13.776651 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 15:23:13.776651 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 15:23:13.777846 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 15:23:13.777846 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:13.777846 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:13.777846 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:13.777846 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:13.777846 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:13.777846 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:13.777846 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:13.777846 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:13.777846 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:13.777846 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:13.777846 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:13.777846 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:13.777846 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:13.777846 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:13.777846 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:13.777846 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:13.778933 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 15:23:13.778933 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 15:23:13.778933 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 15:23:13.778933 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 15:23:13.778933 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 15:23:13.778933 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 15:23:13.778933 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:23:13.778933 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:23:13.782769 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 308/631] 10.222.0.99 () {56 vars in 1111 bytes} [Thu Jul 14 15:23:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:13.784181 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77964ca-c506-4617-bf55-65762d945156 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:13.784474 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77964ca-c506-4617-bf55-65762d945156 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:13.784755 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77964ca-c506-4617-bf55-65762d945156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:13.803303 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-d77964ca-c506-4617-bf55-65762d945156 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:23:13.804598 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d77964ca-c506-4617-bf55-65762d945156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:23:13.848412 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96207371-9010-4a97-8232-8a5a8ca7c4ec None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:13.867759 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:13.883790 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 309/632] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:23:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:23:13.883790 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77964ca-c506-4617-bf55-65762d945156 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:13.897024 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8e7016-07c8-4921-91e4-6c4304fbe08d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:13.898285 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8e7016-07c8-4921-91e4-6c4304fbe08d None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:13.898813 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8e7016-07c8-4921-91e4-6c4304fbe08d None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:13.899297 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8e7016-07c8-4921-91e4-6c4304fbe08d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:13.912062 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8e7016-07c8-4921-91e4-6c4304fbe08d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:13.913696 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8e7016-07c8-4921-91e4-6c4304fbe08d None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:13.933023 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-ab8e7016-07c8-4921-91e4-6c4304fbe08d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:23:13.933023 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:13.933023 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:13.933023 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:13.933023 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:13.933023 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:13.933023 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:13.933023 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:13.933023 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:13.933023 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:13.933023 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:13.933023 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:13.933023 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:23:13.933023 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:23:13.933023 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:23:13.933965 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:23:13.933965 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:13.933965 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:13.933965 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:13.933965 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:13.933965 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:13.933965 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:13.933965 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:13.933965 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:13.933965 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:13.933965 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:13.933965 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:13.933965 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:13.933965 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:13.933965 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:13.933965 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:13.933965 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:13.934828 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:23:13.934828 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:23:13.934828 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:23:13.934828 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:23:13.934828 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 324/633] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:23:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:13.935095 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8e7016-07c8-4921-91e4-6c4304fbe08d None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:14.023623 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77964ca-c506-4617-bf55-65762d945156 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:14.043946 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:14.078723 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8e7016-07c8-4921-91e4-6c4304fbe08d None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:14.082424 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be9ddcb-2628-4999-a45d-e2c2b3949232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:14.083109 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be9ddcb-2628-4999-a45d-e2c2b3949232 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:14.083399 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be9ddcb-2628-4999-a45d-e2c2b3949232 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:14.083717 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be9ddcb-2628-4999-a45d-e2c2b3949232 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:14.084489 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be9ddcb-2628-4999-a45d-e2c2b3949232 None admin] RBAC: Adding query filter params (name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:14.084800 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be9ddcb-2628-4999-a45d-e2c2b3949232 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:14.086223 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be9ddcb-2628-4999-a45d-e2c2b3949232 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:14.102320 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 310/634] 10.222.0.99 () {56 vars in 1103 bytes} [Thu Jul 14 15:23:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:14.102320 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be9ddcb-2628-4999-a45d-e2c2b3949232 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:14.124744 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1331bddd-6da0-4faf-92b8-e0a11730ae56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:14.125901 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1331bddd-6da0-4faf-92b8-e0a11730ae56 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:14.126283 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1331bddd-6da0-4faf-92b8-e0a11730ae56 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:14.126568 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1331bddd-6da0-4faf-92b8-e0a11730ae56 None admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:14.127384 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1331bddd-6da0-4faf-92b8-e0a11730ae56 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:14.127735 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1331bddd-6da0-4faf-92b8-e0a11730ae56 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:14.129250 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1331bddd-6da0-4faf-92b8-e0a11730ae56 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:14.143224 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 325/635] 10.222.0.99 () {56 vars in 1148 bytes} [Thu Jul 14 15:23:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:14.144149 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1331bddd-6da0-4faf-92b8-e0a11730ae56 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:14.166156 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:14.213314 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be9ddcb-2628-4999-a45d-e2c2b3949232 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:14.252002 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425d386a-dbcf-420b-ad3b-7403a1f599be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:14.252798 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425d386a-dbcf-420b-ad3b-7403a1f599be None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:14.253084 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425d386a-dbcf-420b-ad3b-7403a1f599be None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:14.253362 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425d386a-dbcf-420b-ad3b-7403a1f599be None admin] PATH_INFO: `/v3/roles/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:14.261233 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1331bddd-6da0-4faf-92b8-e0a11730ae56 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:14.264314 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425d386a-dbcf-420b-ad3b-7403a1f599be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:23:14.272083 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-425d386a-dbcf-420b-ad3b-7403a1f599be None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 15:23:14.272083 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:14.272083 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:14.272083 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:14.272083 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:14.272083 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:14.272083 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:14.272083 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:14.272083 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:14.272083 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:14.272083 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:14.272083 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:14.272083 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:23:14.272083 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:23:14.272083 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:23:14.272083 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:23:14.273177 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:23:14.273177 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:23:14.273177 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:23:14.273177 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:23:14.273177 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:14.273177 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:14.273177 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:14.273177 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:14.273177 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:14.273177 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:14.273177 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:14.273177 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:14.273177 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:14.273177 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:14.273177 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:14.273177 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:14.274019 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:14.274019 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:14.274019 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:14.274019 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:14.274019 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:23:14.274019 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:23:14.274019 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:23:14.274019 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:23:14.274019 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:23:14.274019 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:23:14.274019 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 15:23:14.274019 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:23:14.274019 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 311/636] 10.222.0.99 () {56 vars in 1102 bytes} [Thu Jul 14 15:23:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:14.274697 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425d386a-dbcf-420b-ad3b-7403a1f599be None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:14.303681 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548c54e6-faa6-48eb-9e1b-a279dbc5c586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:14.304349 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548c54e6-faa6-48eb-9e1b-a279dbc5c586 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:14.304630 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548c54e6-faa6-48eb-9e1b-a279dbc5c586 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:14.304904 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548c54e6-faa6-48eb-9e1b-a279dbc5c586 None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:14.305632 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548c54e6-faa6-48eb-9e1b-a279dbc5c586 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:14.305942 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548c54e6-faa6-48eb-9e1b-a279dbc5c586 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:14.307313 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548c54e6-faa6-48eb-9e1b-a279dbc5c586 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:14.314741 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:14.319905 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 326/637] 10.222.0.99 () {56 vars in 1109 bytes} [Thu Jul 14 15:23:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:14.320877 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548c54e6-faa6-48eb-9e1b-a279dbc5c586 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:14.386567 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425d386a-dbcf-420b-ad3b-7403a1f599be None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:14.412727 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:14.433770 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b7860a-1219-4b86-a611-0c66cc9a9451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:14.434659 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b7860a-1219-4b86-a611-0c66cc9a9451 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:14.435014 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b7860a-1219-4b86-a611-0c66cc9a9451 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:14.435384 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b7860a-1219-4b86-a611-0c66cc9a9451 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:14.441593 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548c54e6-faa6-48eb-9e1b-a279dbc5c586 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:14.446995 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b7860a-1219-4b86-a611-0c66cc9a9451 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:14.448808 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b7860a-1219-4b86-a611-0c66cc9a9451 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:14.466481 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-67b7860a-1219-4b86-a611-0c66cc9a9451 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:23:14.466481 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:14.466481 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:14.466481 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:14.466481 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:14.466481 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:14.466481 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:14.466481 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:14.466481 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:14.466481 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:14.466481 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:14.466481 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:14.466481 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:23:14.466481 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:23:14.466481 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:23:14.468130 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:23:14.468130 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:14.468130 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:14.468130 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:14.468130 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:14.468130 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:14.468130 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:14.468130 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:14.468130 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:14.468130 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:14.468130 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:14.468130 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:14.468130 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:14.468130 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:14.468130 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:14.468130 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:14.468130 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:14.468995 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:23:14.468995 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:23:14.468995 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:23:14.468995 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:23:14.468995 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 312/638] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:23:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:14.469752 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665576a5-469c-4bfc-9f57-a0d2fb1cdfb0 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:14.470042 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665576a5-469c-4bfc-9f57-a0d2fb1cdfb0 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:14.470316 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665576a5-469c-4bfc-9f57-a0d2fb1cdfb0 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:14.471763 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 313/639] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:23:14 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:23:14.472351 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665576a5-469c-4bfc-9f57-a0d2fb1cdfb0 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:14.487798 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7061005c-7527-4938-9ca9-861e11d94e6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:14.488702 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7061005c-7527-4938-9ca9-861e11d94e6c None admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:14.488991 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7061005c-7527-4938-9ca9-861e11d94e6c None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:14.489277 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7061005c-7527-4938-9ca9-861e11d94e6c None admin] PATH_INFO: `/v3/projects/9048d1a6f00443ec92f11471babe5a56/users/ddd2ec3b3ebc4aacb322c95270374451/roles/74e3b597938f49d095cb424c54b5a811` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:14.504582 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7061005c-7527-4938-9ca9-861e11d94e6c None admin] RBAC: Authorizing `identity:create_grant(project_id=9048d1a6f00443ec92f11471babe5a56, user_id=ddd2ec3b3ebc4aacb322c95270374451, role_id=74e3b597938f49d095cb424c54b5a811)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:14.506035 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7061005c-7527-4938-9ca9-861e11d94e6c None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:14.522780 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 327/640] 10.222.0.99 () {58 vars in 1439 bytes} [Thu Jul 14 15:23:14 2022] PUT /identity/v3/projects/9048d1a6f00443ec92f11471babe5a56/users/ddd2ec3b3ebc4aacb322c95270374451/roles/74e3b597938f49d095cb424c54b5a811 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:23:14.524259 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7061005c-7527-4938-9ca9-861e11d94e6c None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:14.591712 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:14.604003 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665576a5-469c-4bfc-9f57-a0d2fb1cdfb0 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:14.647790 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44bcc51-05a9-4f2e-bd46-1aa1fb4d4791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:14.648805 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44bcc51-05a9-4f2e-bd46-1aa1fb4d4791 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:14.649362 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44bcc51-05a9-4f2e-bd46-1aa1fb4d4791 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:14.649641 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44bcc51-05a9-4f2e-bd46-1aa1fb4d4791 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:14.650449 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44bcc51-05a9-4f2e-bd46-1aa1fb4d4791 None admin] RBAC: Adding query filter params (name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:14.650760 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44bcc51-05a9-4f2e-bd46-1aa1fb4d4791 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:14.652213 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44bcc51-05a9-4f2e-bd46-1aa1fb4d4791 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:14.666890 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 314/641] 10.222.0.99 () {56 vars in 1103 bytes} [Thu Jul 14 15:23:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:14.666890 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea38a3c-e694-41fb-917f-8f3de6f51897 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:14.666890 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea38a3c-e694-41fb-917f-8f3de6f51897 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:14.666890 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea38a3c-e694-41fb-917f-8f3de6f51897 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:14.672582 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7061005c-7527-4938-9ca9-861e11d94e6c None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:14.684643 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-cea38a3c-e694-41fb-917f-8f3de6f51897 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:23:14.687196 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-cea38a3c-e694-41fb-917f-8f3de6f51897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:23:14.716911 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88c5981-cca1-40dc-89da-e732947e0eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:14.717810 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88c5981-cca1-40dc-89da-e732947e0eb9 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:14.718294 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88c5981-cca1-40dc-89da-e732947e0eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:14.718726 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88c5981-cca1-40dc-89da-e732947e0eb9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:14.719210 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:14.729555 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88c5981-cca1-40dc-89da-e732947e0eb9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:14.731057 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88c5981-cca1-40dc-89da-e732947e0eb9 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:14.745921 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-f88c5981-cca1-40dc-89da-e732947e0eb9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:23:14.745921 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:14.745921 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:14.745921 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:14.745921 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:14.745921 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:14.745921 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:14.745921 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:14.745921 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:14.745921 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:14.745921 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:14.745921 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:14.745921 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 15:23:14.745921 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 15:23:14.745921 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 15:23:14.746951 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 15:23:14.746951 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:14.746951 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:14.746951 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:14.746951 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:14.746951 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:14.746951 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:14.746951 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:14.746951 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:14.746951 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:14.746951 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:14.746951 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:14.746951 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:14.746951 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:14.746951 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:14.746951 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:14.746951 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:14.748928 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 15:23:14.748928 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 15:23:14.748928 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 15:23:14.748928 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 15:23:14.748928 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 15:23:14.748928 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 15:23:14.748928 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:23:14.748928 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:23:14.748928 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 328/642] 10.222.0.99 () {56 vars in 1111 bytes} [Thu Jul 14 15:23:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:14.748928 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88c5981-cca1-40dc-89da-e732947e0eb9 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:14.785875 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 315/643] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:23:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:23:14.787201 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea38a3c-e694-41fb-917f-8f3de6f51897 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:14.873750 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88c5981-cca1-40dc-89da-e732947e0eb9 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:14.886785 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:14.912121 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea38a3c-e694-41fb-917f-8f3de6f51897 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:14.915756 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e98d694-a67c-4f22-9689-1a3d16e7a713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:14.916817 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e98d694-a67c-4f22-9689-1a3d16e7a713 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:14.917082 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e98d694-a67c-4f22-9689-1a3d16e7a713 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:14.917580 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e98d694-a67c-4f22-9689-1a3d16e7a713 None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:14.918549 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e98d694-a67c-4f22-9689-1a3d16e7a713 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:14.918885 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e98d694-a67c-4f22-9689-1a3d16e7a713 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:14.921028 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e98d694-a67c-4f22-9689-1a3d16e7a713 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:14.934774 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 329/644] 10.222.0.99 () {56 vars in 1109 bytes} [Thu Jul 14 15:23:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:14.935351 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e98d694-a67c-4f22-9689-1a3d16e7a713 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:14.956571 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3edb075-1de5-4db5-92d2-1d51c6c79a04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:14.957393 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3edb075-1de5-4db5-92d2-1d51c6c79a04 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:14.957685 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3edb075-1de5-4db5-92d2-1d51c6c79a04 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:14.958090 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3edb075-1de5-4db5-92d2-1d51c6c79a04 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:14.970660 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3edb075-1de5-4db5-92d2-1d51c6c79a04 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:14.972445 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3edb075-1de5-4db5-92d2-1d51c6c79a04 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:14.995000 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-d3edb075-1de5-4db5-92d2-1d51c6c79a04 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:23:14.995000 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:14.995000 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:14.995000 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:14.995000 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:14.995000 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:14.995000 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:14.995000 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:14.995000 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:14.995000 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:14.995000 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:14.995000 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:14.995000 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:23:14.995000 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:23:14.995000 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:23:14.996009 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:23:14.996009 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:14.996009 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:14.996009 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:14.996009 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:14.996009 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:14.996009 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:14.996009 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:14.996009 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:14.996009 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:14.996009 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:14.996009 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:14.996009 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:14.996009 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:14.996009 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:14.996009 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:14.996009 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:14.997056 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:23:14.997056 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:23:14.997056 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:23:14.997056 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:23:15.001105 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 316/645] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:23:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:15.002031 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3edb075-1de5-4db5-92d2-1d51c6c79a04 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:15.017747 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:15.057655 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e98d694-a67c-4f22-9689-1a3d16e7a713 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:15.097975 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69b5cba-e05a-47f0-89e4-b752541f6ea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:15.098673 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69b5cba-e05a-47f0-89e4-b752541f6ea6 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:15.099208 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69b5cba-e05a-47f0-89e4-b752541f6ea6 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:15.099316 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69b5cba-e05a-47f0-89e4-b752541f6ea6 None admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:15.100104 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69b5cba-e05a-47f0-89e4-b752541f6ea6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:15.100420 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69b5cba-e05a-47f0-89e4-b752541f6ea6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:15.101870 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69b5cba-e05a-47f0-89e4-b752541f6ea6 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:15.115646 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 330/646] 10.222.0.99 () {56 vars in 1148 bytes} [Thu Jul 14 15:23:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:15.116437 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69b5cba-e05a-47f0-89e4-b752541f6ea6 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:15.124610 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3edb075-1de5-4db5-92d2-1d51c6c79a04 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:15.147885 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:15.171640 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d45704-d8e6-4171-b987-eafa0c260e73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:15.172707 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d45704-d8e6-4171-b987-eafa0c260e73 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:15.173167 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d45704-d8e6-4171-b987-eafa0c260e73 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:15.173491 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d45704-d8e6-4171-b987-eafa0c260e73 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:15.190720 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d45704-d8e6-4171-b987-eafa0c260e73 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:23:15.204898 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-52d45704-d8e6-4171-b987-eafa0c260e73 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 15:23:15.204898 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:15.204898 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:15.204898 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:15.204898 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:15.204898 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:15.204898 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:15.204898 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:15.204898 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:15.204898 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:15.204898 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:15.204898 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:15.204898 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:23:15.204898 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:23:15.204898 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:23:15.204898 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:23:15.205928 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:23:15.205928 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:23:15.205928 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:23:15.205928 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:23:15.205928 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:15.205928 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:15.205928 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:15.205928 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:15.205928 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:15.205928 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:15.205928 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:15.205928 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:15.205928 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:15.205928 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:15.205928 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:15.205928 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:15.206795 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:15.206795 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:15.206795 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:15.206795 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:15.206795 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:23:15.206795 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:23:15.206795 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:23:15.206795 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:23:15.206795 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:23:15.206795 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:23:15.206795 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 15:23:15.206795 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:23:15.206795 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 317/647] 10.222.0.99 () {56 vars in 1102 bytes} [Thu Jul 14 15:23:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:15.207991 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d45704-d8e6-4171-b987-eafa0c260e73 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:15.238498 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69b5cba-e05a-47f0-89e4-b752541f6ea6 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:15.277377 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c0f190-9a7b-447b-9d65-8eb7342f65a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:15.278645 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c0f190-9a7b-447b-9d65-8eb7342f65a1 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:15.279130 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c0f190-9a7b-447b-9d65-8eb7342f65a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:15.279525 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c0f190-9a7b-447b-9d65-8eb7342f65a1 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:15.288807 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c0f190-9a7b-447b-9d65-8eb7342f65a1 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:15.290404 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c0f190-9a7b-447b-9d65-8eb7342f65a1 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:15.292327 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:15.308587 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-47c0f190-9a7b-447b-9d65-8eb7342f65a1 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 14 15:23:15.308587 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:15.308587 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:15.308587 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:15.308587 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:15.308587 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:15.308587 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:15.308587 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:15.308587 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:15.308587 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:15.308587 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:15.308587 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:15.308587 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 15:23:15.308587 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 15:23:15.308587 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 15:23:15.308587 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 15:23:15.309589 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:15.309589 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:15.309589 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 15:23:15.309589 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:23:15.309589 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 15:23:15.309589 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:23:15.309589 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:15.309589 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:15.309589 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:15.309589 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:15.309589 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:15.309589 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:15.309589 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:15.309589 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:15.309589 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:15.309589 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:15.309589 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:15.309589 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:15.310516 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:15.310516 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:15.310516 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 15:23:15.310516 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 15:23:15.310516 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 15:23:15.310516 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 15:23:15.310516 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 15:23:15.310516 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 15:23:15.310516 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 14 15:23:15.310516 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:23:15.310516 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 331/648] 10.222.0.99 () {56 vars in 1102 bytes} [Thu Jul 14 15:23:15 2022] GET /identity/v3/users/neutron => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:15.312315 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c0f190-9a7b-447b-9d65-8eb7342f65a1 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:15.333153 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d45704-d8e6-4171-b987-eafa0c260e73 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:15.370692 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47013227-c8f2-4c1a-bf84-910fdb2118ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:15.371355 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47013227-c8f2-4c1a-bf84-910fdb2118ba None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:15.371662 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47013227-c8f2-4c1a-bf84-910fdb2118ba None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:15.371945 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47013227-c8f2-4c1a-bf84-910fdb2118ba None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:15.372666 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47013227-c8f2-4c1a-bf84-910fdb2118ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:15.372977 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47013227-c8f2-4c1a-bf84-910fdb2118ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:15.374616 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47013227-c8f2-4c1a-bf84-910fdb2118ba None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:15.392273 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 318/649] 10.222.0.99 () {56 vars in 1109 bytes} [Thu Jul 14 15:23:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:15.392273 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47013227-c8f2-4c1a-bf84-910fdb2118ba None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:15.422679 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:15.441308 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c0f190-9a7b-447b-9d65-8eb7342f65a1 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:15.486855 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4647785c-0774-4de9-bc7f-ff45b2b34c01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:15.487986 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4647785c-0774-4de9-bc7f-ff45b2b34c01 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:15.488302 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4647785c-0774-4de9-bc7f-ff45b2b34c01 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:15.488584 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4647785c-0774-4de9-bc7f-ff45b2b34c01 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:15.489435 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4647785c-0774-4de9-bc7f-ff45b2b34c01 None admin] RBAC: Adding query filter params (role.id=74e3b597938f49d095cb424c54b5a811, scope.project.id=9048d1a6f00443ec92f11471babe5a56, user.id=605e230d50ce44629f554c95b2f79ead) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:15.489750 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4647785c-0774-4de9-bc7f-ff45b2b34c01 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:15.491615 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4647785c-0774-4de9-bc7f-ff45b2b34c01 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:15.507640 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 332/650] 10.222.0.99 () {56 vars in 1374 bytes} [Thu Jul 14 15:23:15 2022] GET /identity/v3/role_assignments?user.id=605e230d50ce44629f554c95b2f79ead&scope.project.id=9048d1a6f00443ec92f11471babe5a56&role.id=74e3b597938f49d095cb424c54b5a811 => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:15.507640 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4647785c-0774-4de9-bc7f-ff45b2b34c01 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:15.520693 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47013227-c8f2-4c1a-bf84-910fdb2118ba None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:15.572563 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:15.577365 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76789005-52c5-4d77-a2c7-a34aa53789c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:15.578238 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76789005-52c5-4d77-a2c7-a34aa53789c6 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:15.578766 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76789005-52c5-4d77-a2c7-a34aa53789c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:15.579230 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76789005-52c5-4d77-a2c7-a34aa53789c6 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:15.580178 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76789005-52c5-4d77-a2c7-a34aa53789c6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:15.580679 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76789005-52c5-4d77-a2c7-a34aa53789c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:15.583087 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76789005-52c5-4d77-a2c7-a34aa53789c6 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:15.597129 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 319/651] 10.222.0.99 () {56 vars in 1103 bytes} [Thu Jul 14 15:23:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:15.598097 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76789005-52c5-4d77-a2c7-a34aa53789c6 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:15.648008 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4647785c-0774-4de9-bc7f-ff45b2b34c01 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:15.695164 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988b383c-8c72-43c8-a161-e939cdee763e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:15.695903 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988b383c-8c72-43c8-a161-e939cdee763e None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:15.696200 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988b383c-8c72-43c8-a161-e939cdee763e None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:15.696555 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988b383c-8c72-43c8-a161-e939cdee763e None admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:15.697463 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988b383c-8c72-43c8-a161-e939cdee763e None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:15.697788 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988b383c-8c72-43c8-a161-e939cdee763e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:15.699340 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988b383c-8c72-43c8-a161-e939cdee763e None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:15.723140 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 333/652] 10.222.0.99 () {56 vars in 1139 bytes} [Thu Jul 14 15:23:15 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:15.728252 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988b383c-8c72-43c8-a161-e939cdee763e None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:15.741776 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76789005-52c5-4d77-a2c7-a34aa53789c6 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:15.744284 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:15.799898 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28b5c07-e7b6-48cb-bb80-e233b780eeef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:15.801008 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28b5c07-e7b6-48cb-bb80-e233b780eeef None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:15.801302 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28b5c07-e7b6-48cb-bb80-e233b780eeef None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:15.801584 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28b5c07-e7b6-48cb-bb80-e233b780eeef None admin] PATH_INFO: `/v3/users/nova` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:15.814086 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28b5c07-e7b6-48cb-bb80-e233b780eeef None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:15.815098 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28b5c07-e7b6-48cb-bb80-e233b780eeef None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:15.832114 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-d28b5c07-e7b6-48cb-bb80-e233b780eeef None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 14 15:23:15.832114 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:15.832114 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:15.832114 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:15.832114 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:15.832114 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:15.832114 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:15.832114 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:15.832114 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:15.832114 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:15.832114 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:15.832114 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:15.832114 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 15:23:15.832114 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 15:23:15.832114 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 15:23:15.832114 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 15:23:15.833179 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:15.833179 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:15.833179 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 15:23:15.833179 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:23:15.833179 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 15:23:15.833179 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:23:15.833179 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:15.833179 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:15.833179 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:15.833179 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:15.833179 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:15.833179 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:15.833179 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:15.833179 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:15.833179 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:15.833179 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:15.833179 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:15.833179 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:15.834092 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:15.834092 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:15.834092 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 15:23:15.834092 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 15:23:15.834092 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 15:23:15.834092 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 15:23:15.834092 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 15:23:15.834092 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 15:23:15.834092 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 14 15:23:15.834092 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:23:15.834092 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 320/653] 10.222.0.99 () {56 vars in 1093 bytes} [Thu Jul 14 15:23:15 2022] GET /identity/v3/users/nova => generated 82 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:15.834742 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28b5c07-e7b6-48cb-bb80-e233b780eeef None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:15.890662 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988b383c-8c72-43c8-a161-e939cdee763e None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:15.899360 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:15.934852 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d50d25-dda9-45db-a9cd-c7d3b74037ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:15.936860 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d50d25-dda9-45db-a9cd-c7d3b74037ab None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:15.937445 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d50d25-dda9-45db-a9cd-c7d3b74037ab None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:15.937914 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d50d25-dda9-45db-a9cd-c7d3b74037ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:15.951037 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d50d25-dda9-45db-a9cd-c7d3b74037ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:15.953485 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d50d25-dda9-45db-a9cd-c7d3b74037ab None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:15.954390 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28b5c07-e7b6-48cb-bb80-e233b780eeef None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:15.973702 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-b7d50d25-dda9-45db-a9cd-c7d3b74037ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:23:15.973702 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:15.973702 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:15.973702 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:15.973702 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:15.973702 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:15.973702 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:15.973702 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:15.973702 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:15.973702 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:15.973702 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:15.973702 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:15.973702 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:23:15.973702 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:23:15.973702 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:23:15.974644 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:23:15.974644 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:15.974644 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:15.974644 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:15.974644 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:15.974644 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:15.974644 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:15.974644 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:15.974644 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:15.974644 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:15.974644 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:15.974644 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:15.974644 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:15.974644 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:15.974644 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:15.974644 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:15.974644 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:15.975525 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:23:15.975525 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:23:15.975525 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:23:15.975525 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:23:15.982216 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 334/654] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:23:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:15.983417 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d50d25-dda9-45db-a9cd-c7d3b74037ab None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:16.005872 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c89204a-b9dd-4914-a2d1-ac0ed8aebc14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:16.006704 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c89204a-b9dd-4914-a2d1-ac0ed8aebc14 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:16.007060 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c89204a-b9dd-4914-a2d1-ac0ed8aebc14 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:16.007418 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c89204a-b9dd-4914-a2d1-ac0ed8aebc14 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:16.067371 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:16.117668 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d50d25-dda9-45db-a9cd-c7d3b74037ab None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:16.120022 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c89204a-b9dd-4914-a2d1-ac0ed8aebc14 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:16.122585 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c89204a-b9dd-4914-a2d1-ac0ed8aebc14 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:16.157182 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00121a06-5dc6-4708-b356-aa7527f77a4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:16.158250 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00121a06-5dc6-4708-b356-aa7527f77a4d None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:16.158733 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00121a06-5dc6-4708-b356-aa7527f77a4d None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:16.159204 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00121a06-5dc6-4708-b356-aa7527f77a4d None admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:16.160284 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00121a06-5dc6-4708-b356-aa7527f77a4d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:16.160795 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00121a06-5dc6-4708-b356-aa7527f77a4d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:16.162504 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00121a06-5dc6-4708-b356-aa7527f77a4d None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:16.182552 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 335/655] 10.222.0.99 () {56 vars in 1133 bytes} [Thu Jul 14 15:23:15 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:16.183845 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00121a06-5dc6-4708-b356-aa7527f77a4d None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:16.200606 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:16.267824 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 321/656] 10.222.0.99 () {58 vars in 1282 bytes} [Thu Jul 14 15:23:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 15:23:16.269114 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c89204a-b9dd-4914-a2d1-ac0ed8aebc14 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:16.317749 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00121a06-5dc6-4708-b356-aa7527f77a4d None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:16.340709 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:16.360255 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f8bbc1-f9f6-44a9-b300-1805ad03d477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:16.361315 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f8bbc1-f9f6-44a9-b300-1805ad03d477 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:16.361831 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f8bbc1-f9f6-44a9-b300-1805ad03d477 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:16.362307 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f8bbc1-f9f6-44a9-b300-1805ad03d477 None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:16.363249 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f8bbc1-f9f6-44a9-b300-1805ad03d477 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:16.363920 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f8bbc1-f9f6-44a9-b300-1805ad03d477 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:16.365971 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f8bbc1-f9f6-44a9-b300-1805ad03d477 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:16.384387 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 336/657] 10.222.0.99 () {56 vars in 1109 bytes} [Thu Jul 14 15:23:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:16.385888 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f8bbc1-f9f6-44a9-b300-1805ad03d477 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:16.408060 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c89204a-b9dd-4914-a2d1-ac0ed8aebc14 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:16.445726 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1f7838-5fb6-45db-9e48-346919b4cb76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:16.446765 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1f7838-5fb6-45db-9e48-346919b4cb76 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:16.447367 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1f7838-5fb6-45db-9e48-346919b4cb76 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:16.447884 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1f7838-5fb6-45db-9e48-346919b4cb76 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:16.472954 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:16.532657 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f8bbc1-f9f6-44a9-b300-1805ad03d477 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:16.537200 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1f7838-5fb6-45db-9e48-346919b4cb76 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:16.538981 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1f7838-5fb6-45db-9e48-346919b4cb76 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:16.580587 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf397238-fffb-4b50-9598-2e8ced34a58f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:16.581426 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf397238-fffb-4b50-9598-2e8ced34a58f None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:16.581718 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf397238-fffb-4b50-9598-2e8ced34a58f None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:16.582000 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf397238-fffb-4b50-9598-2e8ced34a58f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:16.597798 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf397238-fffb-4b50-9598-2e8ced34a58f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:16.599630 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf397238-fffb-4b50-9598-2e8ced34a58f None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:16.626414 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-cf397238-fffb-4b50-9598-2e8ced34a58f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:23:16.626414 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:16.626414 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:16.626414 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:16.626414 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:16.626414 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:16.626414 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:16.626414 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:16.626414 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:16.626414 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:16.626414 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:16.626414 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:16.626414 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:23:16.626414 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:23:16.626414 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:23:16.627384 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:23:16.627384 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:16.627384 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:16.627384 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:16.627384 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:16.627384 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:16.627384 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:16.627384 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:16.627384 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:16.627384 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:16.627384 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:16.627384 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:16.627384 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:16.627384 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:16.627384 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:16.627384 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:16.627384 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:16.630265 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:23:16.630265 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:23:16.630265 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:23:16.630265 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:23:16.630265 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 337/658] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:23:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:16.630265 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf397238-fffb-4b50-9598-2e8ced34a58f None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:16.667332 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 322/659] 10.222.0.99 () {58 vars in 1282 bytes} [Thu Jul 14 15:23:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 15:23:16.667332 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1f7838-5fb6-45db-9e48-346919b4cb76 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:16.673798 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:16.726310 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:16.751919 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf397238-fffb-4b50-9598-2e8ced34a58f None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:16.781877 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1f7838-5fb6-45db-9e48-346919b4cb76 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:16.790193 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1841f853-51b2-49d6-9ad3-ef1176a8206c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:16.790920 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1841f853-51b2-49d6-9ad3-ef1176a8206c None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:16.791209 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1841f853-51b2-49d6-9ad3-ef1176a8206c None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:16.791489 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1841f853-51b2-49d6-9ad3-ef1176a8206c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:16.800076 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1841f853-51b2-49d6-9ad3-ef1176a8206c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:16.801549 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1841f853-51b2-49d6-9ad3-ef1176a8206c None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:16.818572 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-1841f853-51b2-49d6-9ad3-ef1176a8206c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 14 15:23:16.818572 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:16.818572 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:16.818572 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:16.818572 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:16.818572 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:16.818572 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:16.818572 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:16.818572 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:16.818572 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:16.818572 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:16.818572 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:16.818572 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 15:23:16.818572 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 15:23:16.818572 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 15:23:16.818572 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 15:23:16.819642 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:16.819642 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:16.819642 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 15:23:16.819642 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:23:16.819642 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 15:23:16.819642 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:23:16.819642 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:16.819642 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:16.819642 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:16.819642 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:16.819642 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:16.819642 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:16.819642 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:16.819642 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:16.819642 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:16.819642 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:16.819642 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:16.819642 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:16.820641 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:16.820641 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:16.820641 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 15:23:16.820641 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 15:23:16.820641 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 15:23:16.820641 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 15:23:16.820641 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 15:23:16.820641 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 15:23:16.820641 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 14 15:23:16.820641 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:23:16.823036 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90935eae-adb4-4953-bb0e-da3a5a1136e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:16.823897 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90935eae-adb4-4953-bb0e-da3a5a1136e3 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:16.824028 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90935eae-adb4-4953-bb0e-da3a5a1136e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:16.824234 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 338/660] 10.222.0.99 () {56 vars in 1099 bytes} [Thu Jul 14 15:23:16 2022] GET /identity/v3/users/cinder => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:16.824346 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90935eae-adb4-4953-bb0e-da3a5a1136e3 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:16.825064 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90935eae-adb4-4953-bb0e-da3a5a1136e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:16.825627 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90935eae-adb4-4953-bb0e-da3a5a1136e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:16.825752 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1841f853-51b2-49d6-9ad3-ef1176a8206c None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:16.828182 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90935eae-adb4-4953-bb0e-da3a5a1136e3 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:16.846035 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 323/661] 10.222.0.99 () {56 vars in 1109 bytes} [Thu Jul 14 15:23:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:16.847513 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fb6916-1d13-4707-b6df-d4dcaa7b42dc None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:16.847930 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fb6916-1d13-4707-b6df-d4dcaa7b42dc None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:16.848273 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fb6916-1d13-4707-b6df-d4dcaa7b42dc None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:16.849960 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 324/662] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:23:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:23:16.850661 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fb6916-1d13-4707-b6df-d4dcaa7b42dc None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:16.935309 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:16.944399 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1841f853-51b2-49d6-9ad3-ef1176a8206c None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:16.982153 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fb6916-1d13-4707-b6df-d4dcaa7b42dc None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:16.983449 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692a21cd-8f78-4b50-90dc-ac788e7f09a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:16.983858 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692a21cd-8f78-4b50-90dc-ac788e7f09a6 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:16.984153 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692a21cd-8f78-4b50-90dc-ac788e7f09a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:16.984437 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692a21cd-8f78-4b50-90dc-ac788e7f09a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:16.999759 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:16.999967 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692a21cd-8f78-4b50-90dc-ac788e7f09a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:17.001960 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692a21cd-8f78-4b50-90dc-ac788e7f09a6 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:17.022572 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-692a21cd-8f78-4b50-90dc-ac788e7f09a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:23:17.022572 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:17.022572 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:17.022572 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:17.022572 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:17.022572 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:17.022572 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:17.022572 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:17.022572 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:17.022572 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:17.022572 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:17.022572 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:17.022572 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:23:17.022572 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:23:17.022572 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:23:17.023811 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:23:17.023811 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:17.023811 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:17.023811 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:17.023811 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:17.023811 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:17.023811 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:17.023811 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:17.023811 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:17.023811 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:17.023811 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:17.023811 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:17.023811 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:17.023811 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:17.023811 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:17.023811 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:17.023811 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:17.024678 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:23:17.024678 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:23:17.024678 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:23:17.024678 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:23:17.024678 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 339/663] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:23:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:17.024678 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692a21cd-8f78-4b50-90dc-ac788e7f09a6 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:17.029057 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbc358c-2fb1-4818-8044-b260a634d4a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:17.030288 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbc358c-2fb1-4818-8044-b260a634d4a1 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:17.030583 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbc358c-2fb1-4818-8044-b260a634d4a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:17.030854 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbc358c-2fb1-4818-8044-b260a634d4a1 None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:17.031793 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbc358c-2fb1-4818-8044-b260a634d4a1 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:17.032106 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbc358c-2fb1-4818-8044-b260a634d4a1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:17.033633 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbc358c-2fb1-4818-8044-b260a634d4a1 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:17.054239 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 325/664] 10.222.0.99 () {56 vars in 1137 bytes} [Thu Jul 14 15:23:16 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:17.055849 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d462b656-4fd8-41b1-b8c0-4923d1cf245f None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:17.056344 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d462b656-4fd8-41b1-b8c0-4923d1cf245f None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:17.056931 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d462b656-4fd8-41b1-b8c0-4923d1cf245f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:17.075519 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-d462b656-4fd8-41b1-b8c0-4923d1cf245f None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:23:17.078108 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d462b656-4fd8-41b1-b8c0-4923d1cf245f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:23:17.154096 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692a21cd-8f78-4b50-90dc-ac788e7f09a6 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:17.168023 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 326/665] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:23:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:23:17.168893 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d462b656-4fd8-41b1-b8c0-4923d1cf245f None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:17.207548 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c0236a-8b5d-4442-b912-270574abda9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:17.208717 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c0236a-8b5d-4442-b912-270574abda9e None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:17.209186 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c0236a-8b5d-4442-b912-270574abda9e None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:17.209645 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c0236a-8b5d-4442-b912-270574abda9e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:17.218398 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c0236a-8b5d-4442-b912-270574abda9e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:17.219987 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c0236a-8b5d-4442-b912-270574abda9e None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:17.236955 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-a9c0236a-8b5d-4442-b912-270574abda9e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:23:17.236955 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:17.236955 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:17.236955 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:17.236955 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:17.236955 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:17.236955 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:17.236955 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:17.236955 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:17.236955 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:17.236955 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:17.236955 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:17.236955 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 15:23:17.236955 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 15:23:17.236955 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 15:23:17.237993 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 15:23:17.237993 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:17.237993 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:17.237993 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:17.237993 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:17.237993 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:17.237993 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:17.237993 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:17.237993 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:17.237993 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:17.237993 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:17.237993 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:17.237993 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:17.237993 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:17.237993 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:17.237993 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:17.237993 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:17.238868 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 15:23:17.238868 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 15:23:17.238868 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 15:23:17.238868 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 15:23:17.238868 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 15:23:17.238868 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 15:23:17.238868 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:23:17.238868 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:23:17.238868 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 340/666] 10.222.0.99 () {56 vars in 1111 bytes} [Thu Jul 14 15:23:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:17.239483 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c0236a-8b5d-4442-b912-270574abda9e None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:17.244600 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:17.287537 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d462b656-4fd8-41b1-b8c0-4923d1cf245f None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:17.289530 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:17.329333 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87482857-80d0-44c1-8617-95f3fe171fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:17.329909 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87482857-80d0-44c1-8617-95f3fe171fb8 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:17.330201 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87482857-80d0-44c1-8617-95f3fe171fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:17.330473 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87482857-80d0-44c1-8617-95f3fe171fb8 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:17.331224 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87482857-80d0-44c1-8617-95f3fe171fb8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:17.331508 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87482857-80d0-44c1-8617-95f3fe171fb8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:17.335125 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87482857-80d0-44c1-8617-95f3fe171fb8 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:17.346259 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 327/667] 10.222.0.99 () {56 vars in 1109 bytes} [Thu Jul 14 15:23:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:17.346514 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87482857-80d0-44c1-8617-95f3fe171fb8 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:17.375294 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c0236a-8b5d-4442-b912-270574abda9e None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:17.415832 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b43e69-59d9-49b3-9570-be2b34d82510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:17.416830 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b43e69-59d9-49b3-9570-be2b34d82510 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:17.417304 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b43e69-59d9-49b3-9570-be2b34d82510 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:17.417757 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b43e69-59d9-49b3-9570-be2b34d82510 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:17.465178 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87482857-80d0-44c1-8617-95f3fe171fb8 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:17.495587 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b43e69-59d9-49b3-9570-be2b34d82510 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:17.497425 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b43e69-59d9-49b3-9570-be2b34d82510 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:17.502818 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:17.512997 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d110a73-4ace-4b86-8d1f-7342c027f5e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:17.513848 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d110a73-4ace-4b86-8d1f-7342c027f5e5 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:17.514373 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d110a73-4ace-4b86-8d1f-7342c027f5e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:17.514664 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d110a73-4ace-4b86-8d1f-7342c027f5e5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:17.526483 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d110a73-4ace-4b86-8d1f-7342c027f5e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:23:17.534825 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-3d110a73-4ace-4b86-8d1f-7342c027f5e5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 15:23:17.534825 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:17.534825 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:17.534825 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:17.534825 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:17.534825 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:17.534825 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:17.534825 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:17.534825 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:17.534825 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:17.534825 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:17.534825 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:17.534825 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:23:17.534825 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:23:17.534825 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:23:17.534825 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:23:17.535875 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:23:17.535875 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:23:17.535875 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:23:17.535875 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:23:17.535875 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:17.535875 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:17.535875 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:17.535875 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:17.535875 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:17.535875 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:17.535875 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:17.535875 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:17.535875 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:17.535875 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:17.535875 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:17.535875 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:17.536708 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:17.536708 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:17.536708 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:17.536708 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:17.536708 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:23:17.536708 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:23:17.536708 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:23:17.536708 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:23:17.536708 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:23:17.536708 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:23:17.536708 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 15:23:17.536708 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:23:17.541550 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 328/668] 10.222.0.99 () {56 vars in 1102 bytes} [Thu Jul 14 15:23:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:17.542359 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d110a73-4ace-4b86-8d1f-7342c027f5e5 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:17.571403 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:17.644502 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 341/669] 10.222.0.99 () {58 vars in 1282 bytes} [Thu Jul 14 15:23:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 15:23:17.644502 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b43e69-59d9-49b3-9570-be2b34d82510 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:17.671606 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d110a73-4ace-4b86-8d1f-7342c027f5e5 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:17.715473 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eca27ae-6f8c-47eb-84d7-7d0316fea9ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:17.716299 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eca27ae-6f8c-47eb-84d7-7d0316fea9ac None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:17.716583 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eca27ae-6f8c-47eb-84d7-7d0316fea9ac None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:17.716862 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eca27ae-6f8c-47eb-84d7-7d0316fea9ac None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:17.717738 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eca27ae-6f8c-47eb-84d7-7d0316fea9ac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:17.718051 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eca27ae-6f8c-47eb-84d7-7d0316fea9ac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:17.719509 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eca27ae-6f8c-47eb-84d7-7d0316fea9ac None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:17.734415 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 329/670] 10.222.0.99 () {56 vars in 1148 bytes} [Thu Jul 14 15:23:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:17.735278 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eca27ae-6f8c-47eb-84d7-7d0316fea9ac None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:17.769265 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b43e69-59d9-49b3-9570-be2b34d82510 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:17.772260 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:17.812603 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01660c1f-34dc-4f6a-81a7-81f3a6a7caff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:17.813539 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01660c1f-34dc-4f6a-81a7-81f3a6a7caff None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:17.814089 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01660c1f-34dc-4f6a-81a7-81f3a6a7caff None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:17.814270 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01660c1f-34dc-4f6a-81a7-81f3a6a7caff None admin] PATH_INFO: `/v3/projects/service` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:17.824205 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01660c1f-34dc-4f6a-81a7-81f3a6a7caff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:17.826119 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01660c1f-34dc-4f6a-81a7-81f3a6a7caff None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:17.842401 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-01660c1f-34dc-4f6a-81a7-81f3a6a7caff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:23:17.842401 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:17.842401 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:17.842401 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:17.842401 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:17.842401 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:17.842401 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:17.842401 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:17.842401 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:17.842401 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:17.842401 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:17.842401 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:17.842401 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 15:23:17.842401 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 15:23:17.842401 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 15:23:17.844279 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 15:23:17.844279 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:17.844279 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:17.844279 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:17.844279 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:17.844279 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:17.844279 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:17.844279 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:17.844279 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:17.844279 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:17.844279 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:17.844279 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:17.844279 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:17.844279 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:17.844279 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:17.844279 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:17.844279 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:17.845561 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 15:23:17.845561 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 15:23:17.845561 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 15:23:17.845561 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 15:23:17.845561 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 15:23:17.845561 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 15:23:17.845561 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:23:17.845561 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:23:17.847365 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 342/671] 10.222.0.99 () {56 vars in 1111 bytes} [Thu Jul 14 15:23:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:17.848929 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01660c1f-34dc-4f6a-81a7-81f3a6a7caff None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:17.856297 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:17.880166 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eca27ae-6f8c-47eb-84d7-7d0316fea9ac None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:17.916414 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6d67e0-94d0-4533-99ab-64e978645502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:17.917072 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6d67e0-94d0-4533-99ab-64e978645502 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:17.917383 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6d67e0-94d0-4533-99ab-64e978645502 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:17.917664 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6d67e0-94d0-4533-99ab-64e978645502 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:17.918422 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6d67e0-94d0-4533-99ab-64e978645502 None admin] RBAC: Adding query filter params (name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:17.918732 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6d67e0-94d0-4533-99ab-64e978645502 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:17.920661 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6d67e0-94d0-4533-99ab-64e978645502 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:17.929395 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 330/672] 10.222.0.99 () {56 vars in 1103 bytes} [Thu Jul 14 15:23:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:17.930231 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6d67e0-94d0-4533-99ab-64e978645502 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:17.963302 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01660c1f-34dc-4f6a-81a7-81f3a6a7caff None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:18.003830 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f759793f-2167-44b5-972d-4f234fdd10d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:18.004634 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f759793f-2167-44b5-972d-4f234fdd10d7 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:18.004922 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f759793f-2167-44b5-972d-4f234fdd10d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:18.005201 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f759793f-2167-44b5-972d-4f234fdd10d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:18.016734 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f759793f-2167-44b5-972d-4f234fdd10d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:18.018184 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f759793f-2167-44b5-972d-4f234fdd10d7 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:18.036636 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-f759793f-2167-44b5-972d-4f234fdd10d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:23:18.036636 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:18.036636 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:18.036636 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:18.036636 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:18.036636 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:18.036636 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:18.036636 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:18.036636 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:18.036636 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:18.036636 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:18.036636 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:18.036636 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:23:18.036636 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:23:18.036636 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:23:18.037973 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:23:18.037973 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:18.037973 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:18.037973 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:18.037973 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:18.037973 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:18.037973 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:18.037973 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:18.037973 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:18.037973 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:18.037973 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:18.037973 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:18.037973 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:18.037973 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:18.037973 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:18.037973 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:18.037973 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:18.039032 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:23:18.039032 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:23:18.039032 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:23:18.039032 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:23:18.039032 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 343/673] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:23:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:18.039032 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f759793f-2167-44b5-972d-4f234fdd10d7 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:18.059348 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:18.060589 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6d67e0-94d0-4533-99ab-64e978645502 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:18.098646 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaa2f57-3f8c-4c6d-828c-ca5f95f8e510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:18.099285 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaa2f57-3f8c-4c6d-828c-ca5f95f8e510 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:18.099578 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaa2f57-3f8c-4c6d-828c-ca5f95f8e510 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:18.099892 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaa2f57-3f8c-4c6d-828c-ca5f95f8e510 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:18.100774 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daaa2f57-3f8c-4c6d-828c-ca5f95f8e510 None admin] RBAC: Adding query filter params (role.id=74e3b597938f49d095cb424c54b5a811, scope.project.id=9048d1a6f00443ec92f11471babe5a56, user.id=0d0261759f9743b1a13a760ca9159305) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:18.101054 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daaa2f57-3f8c-4c6d-828c-ca5f95f8e510 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:18.102514 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daaa2f57-3f8c-4c6d-828c-ca5f95f8e510 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:18.108937 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 331/674] 10.222.0.99 () {56 vars in 1374 bytes} [Thu Jul 14 15:23:17 2022] GET /identity/v3/role_assignments?user.id=0d0261759f9743b1a13a760ca9159305&scope.project.id=9048d1a6f00443ec92f11471babe5a56&role.id=74e3b597938f49d095cb424c54b5a811 => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:18.110279 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6673713-58dd-4b27-a08e-6127442cd16a None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:18.110575 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6673713-58dd-4b27-a08e-6127442cd16a None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:18.110860 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6673713-58dd-4b27-a08e-6127442cd16a None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:18.112221 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 332/675] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:23:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:23:18.112951 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6673713-58dd-4b27-a08e-6127442cd16a None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:18.123504 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:18.173471 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f759793f-2167-44b5-972d-4f234fdd10d7 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:18.225462 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a94a96-1691-46bf-a2eb-be88eaba2172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:18.226230 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a94a96-1691-46bf-a2eb-be88eaba2172 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:18.226524 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a94a96-1691-46bf-a2eb-be88eaba2172 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:18.226817 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a94a96-1691-46bf-a2eb-be88eaba2172 None admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:18.227642 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a94a96-1691-46bf-a2eb-be88eaba2172 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:18.227966 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a94a96-1691-46bf-a2eb-be88eaba2172 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:18.229577 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a94a96-1691-46bf-a2eb-be88eaba2172 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:18.243786 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6673713-58dd-4b27-a08e-6127442cd16a None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:18.244573 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 344/676] 10.222.0.99 () {56 vars in 1148 bytes} [Thu Jul 14 15:23:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:18.245552 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a94a96-1691-46bf-a2eb-be88eaba2172 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:18.302748 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7164c4-6f1e-4612-b77b-c77c81f1cf42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:18.303513 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7164c4-6f1e-4612-b77b-c77c81f1cf42 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:18.303881 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7164c4-6f1e-4612-b77b-c77c81f1cf42 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:18.304159 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7164c4-6f1e-4612-b77b-c77c81f1cf42 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:18.318253 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7164c4-6f1e-4612-b77b-c77c81f1cf42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:18.319918 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7164c4-6f1e-4612-b77b-c77c81f1cf42 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:18.341509 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-8f7164c4-6f1e-4612-b77b-c77c81f1cf42 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:23:18.341509 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:18.341509 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:18.341509 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:18.341509 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:18.341509 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:18.341509 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:18.341509 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:18.341509 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:18.341509 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:18.341509 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:18.341509 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:18.341509 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:23:18.341509 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:23:18.341509 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:23:18.343841 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:23:18.343841 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:18.343841 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:18.343841 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:18.343841 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:18.343841 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:18.343841 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:18.343841 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:18.343841 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:18.343841 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:18.343841 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:18.343841 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:18.343841 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:18.343841 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:18.343841 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:18.343841 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:18.343841 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:18.345148 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:23:18.345148 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:23:18.345148 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:23:18.345148 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:23:18.345148 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 333/677] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:23:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:18.345148 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9f2a16-90bb-4c77-8431-f64a27647f8a None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:18.345148 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9f2a16-90bb-4c77-8431-f64a27647f8a None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:18.345148 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9f2a16-90bb-4c77-8431-f64a27647f8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:18.357143 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:18.364345 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-ff9f2a16-90bb-4c77-8431-f64a27647f8a None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:23:18.365578 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ff9f2a16-90bb-4c77-8431-f64a27647f8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:23:18.387055 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a94a96-1691-46bf-a2eb-be88eaba2172 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:18.426059 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:18.438555 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c0f3e5-9de5-4b68-b5e6-24b68b321ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:18.439238 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c0f3e5-9de5-4b68-b5e6-24b68b321ef4 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:18.439528 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c0f3e5-9de5-4b68-b5e6-24b68b321ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:18.439840 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c0f3e5-9de5-4b68-b5e6-24b68b321ef4 None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:18.440579 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c0f3e5-9de5-4b68-b5e6-24b68b321ef4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:18.441115 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c0f3e5-9de5-4b68-b5e6-24b68b321ef4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:18.442265 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c0f3e5-9de5-4b68-b5e6-24b68b321ef4 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:18.452111 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 334/678] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:23:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:23:18.452111 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9f2a16-90bb-4c77-8431-f64a27647f8a None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:18.455974 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 345/679] 10.222.0.99 () {56 vars in 1109 bytes} [Thu Jul 14 15:23:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:18.456973 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c0f3e5-9de5-4b68-b5e6-24b68b321ef4 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:18.577979 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9f2a16-90bb-4c77-8431-f64a27647f8a None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:18.583346 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c0f3e5-9de5-4b68-b5e6-24b68b321ef4 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:18.621309 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277a61eb-5742-49ce-9fc2-146a26f8d144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:18.622077 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277a61eb-5742-49ce-9fc2-146a26f8d144 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:18.622444 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277a61eb-5742-49ce-9fc2-146a26f8d144 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:18.622749 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277a61eb-5742-49ce-9fc2-146a26f8d144 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:18.623858 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277a61eb-5742-49ce-9fc2-146a26f8d144 None admin] RBAC: Adding query filter params (role.id=74e3b597938f49d095cb424c54b5a811, scope.project.id=9048d1a6f00443ec92f11471babe5a56, user.id=3a9b1c2622934c5185f3b5f1ea623bc0) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:18.624538 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277a61eb-5742-49ce-9fc2-146a26f8d144 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:18.625736 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277a61eb-5742-49ce-9fc2-146a26f8d144 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:18.626523 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c367350c-4e99-4028-9db5-cc12edc0a9ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:18.627130 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c367350c-4e99-4028-9db5-cc12edc0a9ef None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:18.627449 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c367350c-4e99-4028-9db5-cc12edc0a9ef None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:18.627921 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c367350c-4e99-4028-9db5-cc12edc0a9ef None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:18.628668 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c367350c-4e99-4028-9db5-cc12edc0a9ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:18.629493 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c367350c-4e99-4028-9db5-cc12edc0a9ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:18.630879 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c367350c-4e99-4028-9db5-cc12edc0a9ef None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:18.632770 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 335/680] 10.222.0.99 () {56 vars in 1374 bytes} [Thu Jul 14 15:23:18 2022] GET /identity/v3/role_assignments?user.id=3a9b1c2622934c5185f3b5f1ea623bc0&scope.project.id=9048d1a6f00443ec92f11471babe5a56&role.id=74e3b597938f49d095cb424c54b5a811 => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:18.633131 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277a61eb-5742-49ce-9fc2-146a26f8d144 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:18.642103 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:18.663197 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 346/681] 10.222.0.99 () {56 vars in 1109 bytes} [Thu Jul 14 15:23:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:18.665242 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c367350c-4e99-4028-9db5-cc12edc0a9ef None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:18.712939 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:18.779401 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277a61eb-5742-49ce-9fc2-146a26f8d144 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:18.826381 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c367350c-4e99-4028-9db5-cc12edc0a9ef None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:18.837208 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9385a3-40f3-4c9d-b922-43f8844661f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:18.838130 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9385a3-40f3-4c9d-b922-43f8844661f6 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:18.838486 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9385a3-40f3-4c9d-b922-43f8844661f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:18.838867 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9385a3-40f3-4c9d-b922-43f8844661f6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:18.847975 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9385a3-40f3-4c9d-b922-43f8844661f6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:18.849584 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9385a3-40f3-4c9d-b922-43f8844661f6 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:18.867534 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-6c9385a3-40f3-4c9d-b922-43f8844661f6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:23:18.867534 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:18.867534 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:18.867534 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:18.867534 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:18.867534 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:18.867534 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:18.867534 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:18.867534 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:18.867534 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:18.867534 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:18.867534 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:18.867534 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 15:23:18.867534 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 15:23:18.867534 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 15:23:18.868769 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 15:23:18.868769 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:18.868769 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:18.868769 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:18.868769 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:18.868769 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:18.868769 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:18.868769 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:18.868769 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:18.868769 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:18.868769 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:18.868769 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:18.868769 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:18.868769 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:18.868769 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:18.868769 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:18.868769 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:18.870069 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 15:23:18.870069 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 15:23:18.870069 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 15:23:18.870069 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 15:23:18.870069 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 15:23:18.870069 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 15:23:18.870069 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:23:18.870069 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:23:18.870069 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 336/682] 10.222.0.99 () {56 vars in 1111 bytes} [Thu Jul 14 15:23:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:18.870069 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9385a3-40f3-4c9d-b922-43f8844661f6 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:18.879854 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1fb224-7d41-4a50-a149-6d9d41238169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:18.880934 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1fb224-7d41-4a50-a149-6d9d41238169 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:18.881393 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1fb224-7d41-4a50-a149-6d9d41238169 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:18.881821 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1fb224-7d41-4a50-a149-6d9d41238169 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:18.894455 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1fb224-7d41-4a50-a149-6d9d41238169 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:18.896325 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1fb224-7d41-4a50-a149-6d9d41238169 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:18.922486 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-eb1fb224-7d41-4a50-a149-6d9d41238169 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:23:18.922486 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:18.922486 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:18.922486 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:18.922486 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:18.922486 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:18.922486 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:18.922486 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:18.922486 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:18.922486 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:18.922486 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:18.922486 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:18.922486 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:23:18.922486 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:23:18.922486 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:23:18.923416 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:23:18.923416 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:18.923416 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:18.923416 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:18.923416 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:18.923416 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:18.923416 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:18.923416 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:18.923416 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:18.923416 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:18.923416 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:18.923416 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:18.923416 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:18.923416 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:18.923416 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:18.923416 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:18.923416 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:18.925220 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:23:18.925220 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:23:18.925220 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:23:18.925220 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:23:18.925220 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 347/683] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:23:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:18.925220 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1fb224-7d41-4a50-a149-6d9d41238169 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:18.995182 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:19.012575 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9385a3-40f3-4c9d-b922-43f8844661f6 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:19.028898 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:19.060493 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a862d30-35bb-4331-a3fe-98ddc18980cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:19.062213 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a862d30-35bb-4331-a3fe-98ddc18980cd None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:19.062759 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a862d30-35bb-4331-a3fe-98ddc18980cd None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:19.063101 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a862d30-35bb-4331-a3fe-98ddc18980cd None admin] PATH_INFO: `/v3/users/swift` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:19.068693 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1fb224-7d41-4a50-a149-6d9d41238169 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:19.077178 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a862d30-35bb-4331-a3fe-98ddc18980cd None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:19.078877 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a862d30-35bb-4331-a3fe-98ddc18980cd None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:19.104309 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-7a862d30-35bb-4331-a3fe-98ddc18980cd None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 14 15:23:19.104309 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:19.104309 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:19.104309 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:19.104309 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:19.104309 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:19.104309 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:19.104309 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:19.104309 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:19.104309 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:19.104309 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:19.104309 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:19.104309 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 15:23:19.104309 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 15:23:19.104309 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 15:23:19.104309 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 15:23:19.105902 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:19.105902 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:19.105902 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 15:23:19.105902 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:23:19.105902 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 15:23:19.105902 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:23:19.105902 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:19.105902 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:19.105902 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:19.105902 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:19.105902 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:19.105902 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:19.105902 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:19.105902 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:19.105902 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:19.105902 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:19.105902 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:19.105902 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:19.107582 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:19.107582 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:19.107582 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 15:23:19.107582 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 15:23:19.107582 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 15:23:19.107582 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 15:23:19.107582 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 15:23:19.107582 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 15:23:19.107582 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 14 15:23:19.107582 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:23:19.109945 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03956ad4-3ea4-40d0-be9b-c6555910a191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:19.110741 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03956ad4-3ea4-40d0-be9b-c6555910a191 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:19.111119 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03956ad4-3ea4-40d0-be9b-c6555910a191 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:19.111486 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03956ad4-3ea4-40d0-be9b-c6555910a191 None admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:19.112405 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03956ad4-3ea4-40d0-be9b-c6555910a191 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:19.112616 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 337/684] 10.222.0.99 () {56 vars in 1096 bytes} [Thu Jul 14 15:23:18 2022] GET /identity/v3/users/swift => generated 83 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:19.112891 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03956ad4-3ea4-40d0-be9b-c6555910a191 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:19.114242 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a862d30-35bb-4331-a3fe-98ddc18980cd None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:19.114701 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03956ad4-3ea4-40d0-be9b-c6555910a191 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:19.128640 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 348/685] 10.222.0.99 () {56 vars in 1148 bytes} [Thu Jul 14 15:23:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:19.129560 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03956ad4-3ea4-40d0-be9b-c6555910a191 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:19.232011 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a862d30-35bb-4331-a3fe-98ddc18980cd None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:19.244111 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03956ad4-3ea4-40d0-be9b-c6555910a191 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:19.253852 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:19.273873 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44378601-3372-47e1-937a-91f4efa0c3e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:19.274921 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44378601-3372-47e1-937a-91f4efa0c3e4 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:19.275359 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44378601-3372-47e1-937a-91f4efa0c3e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:19.275915 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44378601-3372-47e1-937a-91f4efa0c3e4 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:19.276888 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44378601-3372-47e1-937a-91f4efa0c3e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:19.277400 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44378601-3372-47e1-937a-91f4efa0c3e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:19.279000 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44378601-3372-47e1-937a-91f4efa0c3e4 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:19.281900 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf3cf46-38e2-43b2-a97f-20f3bd6f5ab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:19.282837 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf3cf46-38e2-43b2-a97f-20f3bd6f5ab0 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:19.283269 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf3cf46-38e2-43b2-a97f-20f3bd6f5ab0 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:19.283725 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf3cf46-38e2-43b2-a97f-20f3bd6f5ab0 None admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:19.284721 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf3cf46-38e2-43b2-a97f-20f3bd6f5ab0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:19.285177 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf3cf46-38e2-43b2-a97f-20f3bd6f5ab0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:19.286847 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf3cf46-38e2-43b2-a97f-20f3bd6f5ab0 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:19.299627 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 338/686] 10.222.0.99 () {56 vars in 1109 bytes} [Thu Jul 14 15:23:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:19.300440 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44378601-3372-47e1-937a-91f4efa0c3e4 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:19.312512 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 349/687] 10.222.0.99 () {56 vars in 1135 bytes} [Thu Jul 14 15:23:19 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:19.312512 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf3cf46-38e2-43b2-a97f-20f3bd6f5ab0 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:19.352257 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:19.432821 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf3cf46-38e2-43b2-a97f-20f3bd6f5ab0 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:19.455732 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44378601-3372-47e1-937a-91f4efa0c3e4 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:19.480695 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93fcf35-fe9e-4e0b-a693-9e3e54d6c692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:19.482294 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93fcf35-fe9e-4e0b-a693-9e3e54d6c692 None admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:19.482802 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93fcf35-fe9e-4e0b-a693-9e3e54d6c692 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:19.483646 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93fcf35-fe9e-4e0b-a693-9e3e54d6c692 None admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:19.484960 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93fcf35-fe9e-4e0b-a693-9e3e54d6c692 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:19.487288 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93fcf35-fe9e-4e0b-a693-9e3e54d6c692 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:19.493396 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dfd39c-8685-4665-8f8f-b56edd937e59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:19.494142 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dfd39c-8685-4665-8f8f-b56edd937e59 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:19.494496 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dfd39c-8685-4665-8f8f-b56edd937e59 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:19.494861 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dfd39c-8685-4665-8f8f-b56edd937e59 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:19.495811 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dfd39c-8685-4665-8f8f-b56edd937e59 None admin] RBAC: Adding query filter params (role.id=74e3b597938f49d095cb424c54b5a811, scope.project.id=9048d1a6f00443ec92f11471babe5a56, user.id=a3e1a2b83f3d4d88ba5084dfaffd22f3) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:19.496189 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dfd39c-8685-4665-8f8f-b56edd937e59 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:19.497697 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dfd39c-8685-4665-8f8f-b56edd937e59 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:19.505450 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 339/688] 10.222.0.99 () {56 vars in 1374 bytes} [Thu Jul 14 15:23:19 2022] GET /identity/v3/role_assignments?user.id=a3e1a2b83f3d4d88ba5084dfaffd22f3&scope.project.id=9048d1a6f00443ec92f11471babe5a56&role.id=74e3b597938f49d095cb424c54b5a811 => generated 608 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:19.506398 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dfd39c-8685-4665-8f8f-b56edd937e59 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:19.516481 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 350/689] 10.222.0.99 () {60 vars in 1132 bytes} [Thu Jul 14 15:23:19 2022] POST /identity/v3/users => generated 254 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:23:19.674401 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:19.688101 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dfd39c-8685-4665-8f8f-b56edd937e59 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:19.734149 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437b2eb9-a057-41bb-8ccc-dba1aa351dd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:19.734834 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437b2eb9-a057-41bb-8ccc-dba1aa351dd1 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:19.735121 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437b2eb9-a057-41bb-8ccc-dba1aa351dd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:19.735401 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437b2eb9-a057-41bb-8ccc-dba1aa351dd1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:19.832091 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437b2eb9-a057-41bb-8ccc-dba1aa351dd1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:19.834486 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437b2eb9-a057-41bb-8ccc-dba1aa351dd1 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:19.945133 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:19.954475 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 340/690] 10.222.0.99 () {58 vars in 1282 bytes} [Thu Jul 14 15:23:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 15:23:19.960240 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93fcf35-fe9e-4e0b-a693-9e3e54d6c692 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:20.005027 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:20.097930 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93fcf35-fe9e-4e0b-a693-9e3e54d6c692 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:20.134836 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71724164-17d4-420f-ba29-65835e4b56c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:20.135921 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71724164-17d4-420f-ba29-65835e4b56c5 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:20.136360 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71724164-17d4-420f-ba29-65835e4b56c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:20.136811 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71724164-17d4-420f-ba29-65835e4b56c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:20.148449 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71724164-17d4-420f-ba29-65835e4b56c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:20.150229 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71724164-17d4-420f-ba29-65835e4b56c5 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:20.169058 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-71724164-17d4-420f-ba29-65835e4b56c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:23:20.169058 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:20.169058 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:20.169058 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:20.169058 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:20.169058 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:20.169058 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:20.169058 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:20.169058 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:20.169058 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:20.169058 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:20.169058 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:20.169058 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:23:20.169058 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:23:20.169058 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:23:20.170221 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:23:20.170221 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:20.170221 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:20.170221 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:20.170221 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:20.170221 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:20.170221 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:20.170221 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:20.170221 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:20.170221 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:20.170221 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:20.170221 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:20.170221 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:20.170221 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:20.170221 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:20.170221 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:20.170221 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:20.171861 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:23:20.171861 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:23:20.171861 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:23:20.171861 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:23:20.173817 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 351/691] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:23:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:20.185481 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437b2eb9-a057-41bb-8ccc-dba1aa351dd1 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:20.299382 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437b2eb9-a057-41bb-8ccc-dba1aa351dd1 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:20.337556 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb4a045-d7c6-4793-90f0-a168da996e84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:20.338485 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb4a045-d7c6-4793-90f0-a168da996e84 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:20.338948 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb4a045-d7c6-4793-90f0-a168da996e84 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:20.339378 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb4a045-d7c6-4793-90f0-a168da996e84 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:20.340291 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb4a045-d7c6-4793-90f0-a168da996e84 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:20.340752 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb4a045-d7c6-4793-90f0-a168da996e84 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:20.342406 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb4a045-d7c6-4793-90f0-a168da996e84 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:20.353969 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51300cc5-e820-4b32-88f6-708231556385 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:20.355238 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51300cc5-e820-4b32-88f6-708231556385 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:20.355768 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51300cc5-e820-4b32-88f6-708231556385 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:20.357260 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 341/692] 10.222.0.99 () {56 vars in 1109 bytes} [Thu Jul 14 15:23:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:20.357816 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 352/693] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:23:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 15:23:20.362666 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb4a045-d7c6-4793-90f0-a168da996e84 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:20.364508 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522eebd3-d3d4-44e3-9974-f4fbee2e5d1f None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:20.365034 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522eebd3-d3d4-44e3-9974-f4fbee2e5d1f None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:20.365785 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522eebd3-d3d4-44e3-9974-f4fbee2e5d1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:20.387070 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-522eebd3-d3d4-44e3-9974-f4fbee2e5d1f None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:23:20.388861 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-522eebd3-d3d4-44e3-9974-f4fbee2e5d1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:23:20.482004 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:20.515996 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 353/694] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:23:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:23:20.569751 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb4a045-d7c6-4793-90f0-a168da996e84 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:20.573480 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522eebd3-d3d4-44e3-9974-f4fbee2e5d1f None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:20.576274 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:20.612020 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba83f14-e312-4b79-9c49-35bfac4a031b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:20.613187 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba83f14-e312-4b79-9c49-35bfac4a031b None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:20.613683 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba83f14-e312-4b79-9c49-35bfac4a031b None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:20.614257 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba83f14-e312-4b79-9c49-35bfac4a031b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:20.622329 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba83f14-e312-4b79-9c49-35bfac4a031b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:20.624772 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba83f14-e312-4b79-9c49-35bfac4a031b None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:20.642250 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-eba83f14-e312-4b79-9c49-35bfac4a031b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:23:20.642250 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:20.642250 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:20.642250 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:20.642250 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:20.642250 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:20.642250 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:20.642250 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:20.642250 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:20.642250 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:20.642250 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:20.642250 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:20.642250 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 15:23:20.642250 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 15:23:20.642250 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 15:23:20.643199 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 15:23:20.643199 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:20.643199 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:20.643199 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:20.643199 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:20.643199 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:20.643199 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:20.643199 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:20.643199 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:20.643199 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:20.643199 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:20.643199 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:20.643199 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:20.643199 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:20.643199 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:20.643199 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:20.643199 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:20.644126 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 15:23:20.644126 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 15:23:20.644126 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 15:23:20.644126 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 15:23:20.644126 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 15:23:20.644126 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 15:23:20.644126 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:23:20.644126 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:23:20.646623 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 342/695] 10.222.0.99 () {56 vars in 1111 bytes} [Thu Jul 14 15:23:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 15:23:20.657013 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba83f14-e312-4b79-9c49-35bfac4a031b None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:20.720048 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522eebd3-d3d4-44e3-9974-f4fbee2e5d1f None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:20.760780 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43288e2-82df-43b3-842b-d807dd79111b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:20.761592 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43288e2-82df-43b3-842b-d807dd79111b None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:20.761874 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43288e2-82df-43b3-842b-d807dd79111b None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:20.762147 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43288e2-82df-43b3-842b-d807dd79111b None admin] PATH_INFO: `/v3/services/network` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:20.762818 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43288e2-82df-43b3-842b-d807dd79111b None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:20.764766 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43288e2-82df-43b3-842b-d807dd79111b None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:20.782780 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-e43288e2-82df-43b3-842b-d807dd79111b None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 14 15:23:20.782780 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:20.782780 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 15:23:20.782780 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 15:23:20.782780 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 15:23:20.782780 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 15:23:20.782780 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 15:23:20.782780 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 15:23:20.782780 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 14 15:23:20.782780 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:23:20.782780 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 15:23:20.782780 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:23:20.782780 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:20.782780 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:20.782780 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:20.782780 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:20.782780 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:20.783913 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:20.783913 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:20.783913 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:20.783913 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:20.783913 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:20.783913 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:20.783913 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 15:23:20.783913 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 15:23:20.783913 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 15:23:20.783913 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 15:23:20.783913 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:20.783913 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:20.783913 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:20.783913 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:20.783913 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:20.783913 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:20.784763 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:20.784763 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:20.784763 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:20.784763 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:20.784763 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:20.784763 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:20.784763 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:20.784763 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:20.784763 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:20.784763 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:20.784763 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 15:23:20.784763 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 15:23:20.784763 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 14 15:23:20.784763 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:23:20.784763 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 354/696] 10.222.0.99 () {56 vars in 1111 bytes} [Thu Jul 14 15:23:20 2022] GET /identity/v3/services/network => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:20.792612 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43288e2-82df-43b3-842b-d807dd79111b None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:20.887140 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:20.925416 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba83f14-e312-4b79-9c49-35bfac4a031b None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:20.939821 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43288e2-82df-43b3-842b-d807dd79111b None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:20.962293 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:21.003893 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19151890-8989-49be-a86f-a791a6df157d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:21.003893 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19151890-8989-49be-a86f-a791a6df157d None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:21.003893 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19151890-8989-49be-a86f-a791a6df157d None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:21.003893 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19151890-8989-49be-a86f-a791a6df157d None admin] PATH_INFO: `/v3/services` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:21.004286 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19151890-8989-49be-a86f-a791a6df157d None admin] RBAC: Adding query filter params (name=network) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:21.004286 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19151890-8989-49be-a86f-a791a6df157d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:21.004286 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19151890-8989-49be-a86f-a791a6df157d None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:21.004494 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4dcdae-4519-4e1f-a262-78f37165aa17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:21.004839 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4dcdae-4519-4e1f-a262-78f37165aa17 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:21.005125 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4dcdae-4519-4e1f-a262-78f37165aa17 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:21.005402 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4dcdae-4519-4e1f-a262-78f37165aa17 None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:21.006207 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4dcdae-4519-4e1f-a262-78f37165aa17 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:21.006518 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4dcdae-4519-4e1f-a262-78f37165aa17 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:21.007969 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4dcdae-4519-4e1f-a262-78f37165aa17 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:21.016775 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 355/697] 10.222.0.99 () {56 vars in 1112 bytes} [Thu Jul 14 15:23:20 2022] GET /identity/v3/services?name=network => generated 124 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:21.022762 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19151890-8989-49be-a86f-a791a6df157d None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:21.041199 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 343/698] 10.222.0.99 () {56 vars in 1148 bytes} [Thu Jul 14 15:23:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 384 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:21.047713 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4dcdae-4519-4e1f-a262-78f37165aa17 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:21.200360 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19151890-8989-49be-a86f-a791a6df157d None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:21.240098 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df430d2-feb4-4e28-807f-599d83be6b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:21.240790 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df430d2-feb4-4e28-807f-599d83be6b78 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:21.241085 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df430d2-feb4-4e28-807f-599d83be6b78 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:21.241369 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df430d2-feb4-4e28-807f-599d83be6b78 None admin] PATH_INFO: `/v3/services` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:21.242171 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df430d2-feb4-4e28-807f-599d83be6b78 None admin] RBAC: Adding query filter params (type=network) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:21.242584 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df430d2-feb4-4e28-807f-599d83be6b78 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:21.244157 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df430d2-feb4-4e28-807f-599d83be6b78 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:21.247999 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:21.251349 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 356/699] 10.222.0.99 () {56 vars in 1112 bytes} [Thu Jul 14 15:23:21 2022] GET /identity/v3/services?type=network => generated 124 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:21.275497 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4dcdae-4519-4e1f-a262-78f37165aa17 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:21.316863 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80550073-c6f1-4653-ae8c-efa9c9e4a0e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:21.317735 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80550073-c6f1-4653-ae8c-efa9c9e4a0e5 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:21.318113 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80550073-c6f1-4653-ae8c-efa9c9e4a0e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:21.318487 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80550073-c6f1-4653-ae8c-efa9c9e4a0e5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:21.319453 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80550073-c6f1-4653-ae8c-efa9c9e4a0e5 None admin] RBAC: Adding query filter params (role.id=74e3b597938f49d095cb424c54b5a811, scope.project.id=9048d1a6f00443ec92f11471babe5a56, user.id=ddd2ec3b3ebc4aacb322c95270374451) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:21.319891 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80550073-c6f1-4653-ae8c-efa9c9e4a0e5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:21.321450 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80550073-c6f1-4653-ae8c-efa9c9e4a0e5 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:21.329550 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 344/700] 10.222.0.99 () {56 vars in 1374 bytes} [Thu Jul 14 15:23:21 2022] GET /identity/v3/role_assignments?user.id=ddd2ec3b3ebc4aacb322c95270374451&scope.project.id=9048d1a6f00443ec92f11471babe5a56&role.id=74e3b597938f49d095cb424c54b5a811 => generated 608 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:21.412749 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fde02f2-7c19-4f47-8e38-15f37dd55af3 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:21.412749 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fde02f2-7c19-4f47-8e38-15f37dd55af3 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:21.412749 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fde02f2-7c19-4f47-8e38-15f37dd55af3 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:21.415398 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 357/701] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:23:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:23:21.421697 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8006d8-9810-4101-9d0c-a228c926701d None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:21.422064 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8006d8-9810-4101-9d0c-a228c926701d None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:21.422411 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8006d8-9810-4101-9d0c-a228c926701d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:21.445539 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-bf8006d8-9810-4101-9d0c-a228c926701d None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:23:21.446759 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-bf8006d8-9810-4101-9d0c-a228c926701d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:23:21.505778 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:21.563968 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 345/702] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:23:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:23:21.581070 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fde02f2-7c19-4f47-8e38-15f37dd55af3 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:21.719242 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fde02f2-7c19-4f47-8e38-15f37dd55af3 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:21.759817 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94dc73e-d890-4591-982b-39cde6ffc94e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:21.760622 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94dc73e-d890-4591-982b-39cde6ffc94e None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:21.760907 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94dc73e-d890-4591-982b-39cde6ffc94e None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:21.761178 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94dc73e-d890-4591-982b-39cde6ffc94e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:21.774014 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94dc73e-d890-4591-982b-39cde6ffc94e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:23:21.785250 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-b94dc73e-d890-4591-982b-39cde6ffc94e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:23:21.785250 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:21.785250 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:21.785250 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:21.785250 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:21.785250 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:21.785250 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:21.785250 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:21.785250 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:21.785250 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:21.785250 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:21.785250 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:21.785250 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:23:21.785250 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:23:21.785250 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:23:21.785250 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:23:21.787640 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:23:21.787640 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:23:21.787640 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:23:21.787640 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:23:21.787640 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:21.787640 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:21.787640 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:21.787640 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:21.787640 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:21.787640 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:21.787640 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:21.787640 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:21.787640 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:21.787640 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:21.787640 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:21.787640 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:21.789732 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:21.789732 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:21.789732 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:21.789732 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:21.789732 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:23:21.789732 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:23:21.789732 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:23:21.789732 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:23:21.789732 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:23:21.789732 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:23:21.789732 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:23:21.789732 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:23:21.789732 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 358/703] 10.222.0.99 () {56 vars in 1096 bytes} [Thu Jul 14 15:23:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:21.799083 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8006d8-9810-4101-9d0c-a228c926701d None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:21.912910 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8006d8-9810-4101-9d0c-a228c926701d None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:21.954254 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eec9c0-4587-48d2-9984-ee3f27fe01bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:21.955193 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eec9c0-4587-48d2-9984-ee3f27fe01bf None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:21.955667 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eec9c0-4587-48d2-9984-ee3f27fe01bf None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:21.956107 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eec9c0-4587-48d2-9984-ee3f27fe01bf None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:21.957049 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49eec9c0-4587-48d2-9984-ee3f27fe01bf None admin] RBAC: Adding query filter params (name=admin) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:21.957544 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49eec9c0-4587-48d2-9984-ee3f27fe01bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:21.959057 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49eec9c0-4587-48d2-9984-ee3f27fe01bf None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:21.969159 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 346/704] 10.222.0.99 () {56 vars in 1099 bytes} [Thu Jul 14 15:23:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:21.975111 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94dc73e-d890-4591-982b-39cde6ffc94e None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:22.042003 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:22.091876 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94dc73e-d890-4591-982b-39cde6ffc94e None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:22.137615 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec59d670-f509-4dd1-bb03-69609c85b32b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:22.138428 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec59d670-f509-4dd1-bb03-69609c85b32b None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:22.138711 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec59d670-f509-4dd1-bb03-69609c85b32b None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:22.138987 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec59d670-f509-4dd1-bb03-69609c85b32b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:22.151082 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec59d670-f509-4dd1-bb03-69609c85b32b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:22.152663 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec59d670-f509-4dd1-bb03-69609c85b32b None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:22.171725 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-ec59d670-f509-4dd1-bb03-69609c85b32b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:23:22.171725 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:22.171725 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:22.171725 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:22.171725 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:22.171725 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:22.171725 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:22.171725 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:22.171725 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:22.171725 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:22.171725 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:22.171725 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:22.171725 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:23:22.171725 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:23:22.171725 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:23:22.172688 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:23:22.172688 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:22.172688 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:22.172688 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:22.172688 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:22.172688 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:22.172688 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:22.172688 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:22.172688 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:22.172688 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:22.172688 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:22.172688 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:22.172688 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:22.172688 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:22.172688 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:22.172688 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:22.172688 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:22.173575 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:23:22.173575 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:23:22.173575 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:23:22.173575 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:23:22.173575 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 359/705] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:23:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:22.179030 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eec9c0-4587-48d2-9984-ee3f27fe01bf None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:22.189686 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0097f01-79db-4b59-bce9-0d5bf4c11111 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:22.190041 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0097f01-79db-4b59-bce9-0d5bf4c11111 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:22.190252 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0097f01-79db-4b59-bce9-0d5bf4c11111 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:22.193928 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 360/706] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:23:22 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:23:22.200026 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e731522-0c6b-4814-b65d-9b74ba92f186 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:22.200318 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e731522-0c6b-4814-b65d-9b74ba92f186 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:22.200605 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e731522-0c6b-4814-b65d-9b74ba92f186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:22.218697 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-3e731522-0c6b-4814-b65d-9b74ba92f186 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:23:22.220023 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-3e731522-0c6b-4814-b65d-9b74ba92f186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:23:22.242435 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:22.313688 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 361/707] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:23:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:23:22.315143 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaa44bb-e836-41ab-ada8-7fceb094d795 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:22.315434 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaa44bb-e836-41ab-ada8-7fceb094d795 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:22.315742 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaa44bb-e836-41ab-ada8-7fceb094d795 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:22.317364 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 362/708] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:23:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:23:22.326651 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244da950-1bd5-43c3-aaba-ac122964dcbd None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:22.326946 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244da950-1bd5-43c3-aaba-ac122964dcbd None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:22.327892 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244da950-1bd5-43c3-aaba-ac122964dcbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:22.351870 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eec9c0-4587-48d2-9984-ee3f27fe01bf None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:22.355334 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-244da950-1bd5-43c3-aaba-ac122964dcbd None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:23:22.363645 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-244da950-1bd5-43c3-aaba-ac122964dcbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:23:22.390171 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c7e887-d39b-4a4d-a346-ce4019e2125b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:22.391030 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c7e887-d39b-4a4d-a346-ce4019e2125b None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:22.391547 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:22.392490 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c7e887-d39b-4a4d-a346-ce4019e2125b None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:22.392821 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c7e887-d39b-4a4d-a346-ce4019e2125b None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:22.393865 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c7e887-d39b-4a4d-a346-ce4019e2125b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:22.394207 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c7e887-d39b-4a4d-a346-ce4019e2125b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:22.396653 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c7e887-d39b-4a4d-a346-ce4019e2125b None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:22.412978 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 347/709] 10.222.0.99 () {56 vars in 1109 bytes} [Thu Jul 14 15:23:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:22.413888 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c7e887-d39b-4a4d-a346-ce4019e2125b None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:22.439712 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 363/710] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:23:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:23:22.440681 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244da950-1bd5-43c3-aaba-ac122964dcbd None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:22.520010 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:22.529615 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c7e887-d39b-4a4d-a346-ce4019e2125b None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:22.554930 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244da950-1bd5-43c3-aaba-ac122964dcbd None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:22.569771 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f36798f-2d86-4bbb-9f63-5a64ce17c5a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:22.570621 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f36798f-2d86-4bbb-9f63-5a64ce17c5a5 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:22.570984 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f36798f-2d86-4bbb-9f63-5a64ce17c5a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:22.571452 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f36798f-2d86-4bbb-9f63-5a64ce17c5a5 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:22.572231 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f36798f-2d86-4bbb-9f63-5a64ce17c5a5 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:22.573702 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f36798f-2d86-4bbb-9f63-5a64ce17c5a5 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:22.589703 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-8f36798f-2d86-4bbb-9f63-5a64ce17c5a5 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 14 15:23:22.589703 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:22.589703 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 15:23:22.589703 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 15:23:22.589703 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 15:23:22.589703 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 15:23:22.589703 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 15:23:22.589703 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 15:23:22.589703 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 14 15:23:22.589703 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:23:22.589703 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 15:23:22.589703 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:23:22.589703 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:22.589703 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:22.589703 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:22.589703 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:22.590972 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:22.590972 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:22.590972 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:22.590972 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:22.590972 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:22.590972 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:22.590972 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:22.590972 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 15:23:22.590972 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 15:23:22.590972 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 15:23:22.590972 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 15:23:22.590972 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:22.590972 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:22.590972 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:22.590972 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:22.590972 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:22.592069 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:22.592069 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:22.592069 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:22.592069 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:22.592069 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:22.592069 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:22.592069 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:22.592069 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:22.592069 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:22.592069 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:22.592069 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:22.592069 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 15:23:22.592069 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 15:23:22.592069 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 14 15:23:22.592069 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:23:22.598265 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 348/711] 10.222.0.99 () {56 vars in 1129 bytes} [Thu Jul 14 15:23:22 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:22.598835 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03b21de-67eb-45e8-865e-c4df711dac40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:22.599328 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f36798f-2d86-4bbb-9f63-5a64ce17c5a5 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:22.600357 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03b21de-67eb-45e8-865e-c4df711dac40 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:22.600912 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03b21de-67eb-45e8-865e-c4df711dac40 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:22.601465 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03b21de-67eb-45e8-865e-c4df711dac40 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:22.613262 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03b21de-67eb-45e8-865e-c4df711dac40 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:22.615056 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03b21de-67eb-45e8-865e-c4df711dac40 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:22.632875 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-b03b21de-67eb-45e8-865e-c4df711dac40 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 14 15:23:22.632875 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:22.632875 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:22.632875 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:22.632875 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:22.632875 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:22.632875 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:22.632875 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:22.632875 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:22.632875 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:22.632875 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:22.632875 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:22.632875 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 15:23:22.632875 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 15:23:22.632875 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 15:23:22.632875 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 15:23:22.634547 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:22.634547 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:22.634547 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 15:23:22.634547 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:23:22.634547 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 15:23:22.634547 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:23:22.634547 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:22.634547 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:22.634547 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:22.634547 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:22.634547 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:22.634547 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:22.634547 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:22.634547 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:22.634547 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:22.634547 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:22.634547 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:22.634547 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:22.636758 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:22.636758 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:22.636758 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 15:23:22.636758 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 15:23:22.636758 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 15:23:22.636758 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 15:23:22.636758 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 15:23:22.636758 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 15:23:22.636758 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 14 15:23:22.636758 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:23:22.636758 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 364/712] 10.222.0.99 () {56 vars in 1093 bytes} [Thu Jul 14 15:23:22 2022] GET /identity/v3/users/nova => generated 82 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:22.641692 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03b21de-67eb-45e8-865e-c4df711dac40 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:22.674289 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:22.715583 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f36798f-2d86-4bbb-9f63-5a64ce17c5a5 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:22.757822 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cf329b-2ea6-4a20-96d5-023969df1792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:22.758687 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cf329b-2ea6-4a20-96d5-023969df1792 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:22.758920 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cf329b-2ea6-4a20-96d5-023969df1792 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:22.759465 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cf329b-2ea6-4a20-96d5-023969df1792 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:22.771435 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cf329b-2ea6-4a20-96d5-023969df1792 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:23:22.772136 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03b21de-67eb-45e8-865e-c4df711dac40 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:22.779421 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-80cf329b-2ea6-4a20-96d5-023969df1792 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 15:23:22.779421 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:22.779421 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:22.779421 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:22.779421 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:22.779421 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:22.779421 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:22.779421 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:22.779421 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:22.779421 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:22.779421 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:22.779421 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:22.779421 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:23:22.779421 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:23:22.779421 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:23:22.779421 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:23:22.780605 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:23:22.780605 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:23:22.780605 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:23:22.780605 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:23:22.780605 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:22.780605 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:22.780605 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:22.780605 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:22.780605 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:22.780605 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:22.780605 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:22.780605 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:22.780605 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:22.780605 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:22.780605 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:22.780605 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:22.781477 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:22.781477 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:22.781477 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:22.781477 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:22.781477 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:23:22.781477 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:23:22.781477 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:23:22.781477 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:23:22.781477 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:23:22.781477 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:23:22.781477 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 15:23:22.781477 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:23:22.781477 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 349/713] 10.222.0.99 () {56 vars in 1102 bytes} [Thu Jul 14 15:23:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:22.782188 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cf329b-2ea6-4a20-96d5-023969df1792 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:22.803206 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:22.817120 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041a1a51-f637-4beb-9e97-41bddfcbacd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:22.818204 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041a1a51-f637-4beb-9e97-41bddfcbacd5 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:22.818677 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041a1a51-f637-4beb-9e97-41bddfcbacd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:22.819309 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041a1a51-f637-4beb-9e97-41bddfcbacd5 None admin] PATH_INFO: `/v3/services` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:22.820332 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041a1a51-f637-4beb-9e97-41bddfcbacd5 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:22.820868 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041a1a51-f637-4beb-9e97-41bddfcbacd5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:22.822615 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041a1a51-f637-4beb-9e97-41bddfcbacd5 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:22.828253 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 365/714] 10.222.0.99 () {56 vars in 1124 bytes} [Thu Jul 14 15:23:22 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:22.829483 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041a1a51-f637-4beb-9e97-41bddfcbacd5 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:22.898984 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cf329b-2ea6-4a20-96d5-023969df1792 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:22.937069 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e768bd8-034d-417c-b990-6b68882d9c2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:22.937748 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e768bd8-034d-417c-b990-6b68882d9c2a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:22.938039 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e768bd8-034d-417c-b990-6b68882d9c2a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:22.938323 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e768bd8-034d-417c-b990-6b68882d9c2a None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:22.939179 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e768bd8-034d-417c-b990-6b68882d9c2a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:22.939495 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e768bd8-034d-417c-b990-6b68882d9c2a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:22.941006 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e768bd8-034d-417c-b990-6b68882d9c2a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:22.944772 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:22.956105 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041a1a51-f637-4beb-9e97-41bddfcbacd5 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:22.964845 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 350/715] 10.222.0.99 () {56 vars in 1133 bytes} [Thu Jul 14 15:23:22 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:22.964845 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e768bd8-034d-417c-b990-6b68882d9c2a None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:23.002925 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec75e98-85c0-49c7-b63c-68a14c1ecd51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:23.004021 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec75e98-85c0-49c7-b63c-68a14c1ecd51 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:23.004674 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec75e98-85c0-49c7-b63c-68a14c1ecd51 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:23.005267 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec75e98-85c0-49c7-b63c-68a14c1ecd51 None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:23.006968 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec75e98-85c0-49c7-b63c-68a14c1ecd51 None admin] RBAC: Adding query filter params (name=service) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:23.007534 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec75e98-85c0-49c7-b63c-68a14c1ecd51 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:23.009452 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec75e98-85c0-49c7-b63c-68a14c1ecd51 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:23.019841 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 366/716] 10.222.0.99 () {56 vars in 1103 bytes} [Thu Jul 14 15:23:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:23.020995 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec75e98-85c0-49c7-b63c-68a14c1ecd51 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:23.052247 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:23.083273 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e768bd8-034d-417c-b990-6b68882d9c2a None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:23.122297 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c7abd3-eebe-42a9-a358-f1ad8a08ab48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:23.122987 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c7abd3-eebe-42a9-a358-f1ad8a08ab48 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:23.123279 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c7abd3-eebe-42a9-a358-f1ad8a08ab48 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:23.123620 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c7abd3-eebe-42a9-a358-f1ad8a08ab48 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:23.124440 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c7abd3-eebe-42a9-a358-f1ad8a08ab48 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:23.124752 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c7abd3-eebe-42a9-a358-f1ad8a08ab48 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:23.126272 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c7abd3-eebe-42a9-a358-f1ad8a08ab48 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:23.139967 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 351/717] 10.222.0.99 () {56 vars in 1124 bytes} [Thu Jul 14 15:23:22 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:23.141020 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c7abd3-eebe-42a9-a358-f1ad8a08ab48 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:23.156035 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec75e98-85c0-49c7-b63c-68a14c1ecd51 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:23.197618 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dca1b36-97e0-4113-84a0-1b2e18698961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:23.198857 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dca1b36-97e0-4113-84a0-1b2e18698961 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:23.199391 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dca1b36-97e0-4113-84a0-1b2e18698961 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:23.199952 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dca1b36-97e0-4113-84a0-1b2e18698961 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:23.230358 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:23.288696 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dca1b36-97e0-4113-84a0-1b2e18698961 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:23.290665 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dca1b36-97e0-4113-84a0-1b2e18698961 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:23.329948 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c7abd3-eebe-42a9-a358-f1ad8a08ab48 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:23.373886 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4bcbba-3fb6-4a2a-8508-e1a9e4bb5911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:23.375373 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4bcbba-3fb6-4a2a-8508-e1a9e4bb5911 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:23.375499 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4bcbba-3fb6-4a2a-8508-e1a9e4bb5911 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:23.375820 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4bcbba-3fb6-4a2a-8508-e1a9e4bb5911 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:23.384094 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:23.389860 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4bcbba-3fb6-4a2a-8508-e1a9e4bb5911 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:23.392211 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4bcbba-3fb6-4a2a-8508-e1a9e4bb5911 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:23.413039 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-4b4bcbba-3fb6-4a2a-8508-e1a9e4bb5911 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:23:23.413039 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:23.413039 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:23.413039 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:23.413039 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:23.413039 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:23.413039 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:23.413039 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:23.413039 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:23.413039 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:23.413039 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:23.413039 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:23.413039 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:23:23.413039 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:23:23.413039 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:23:23.414240 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:23:23.414240 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:23.414240 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:23.414240 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:23.414240 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:23.414240 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:23.414240 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:23.414240 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:23.414240 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:23.414240 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:23.414240 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:23.414240 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:23.414240 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:23.414240 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:23.414240 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:23.414240 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:23.414240 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:23.415159 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:23:23.415159 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:23:23.415159 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:23:23.415159 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:23:23.415801 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 352/718] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:23:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:23.421658 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 367/719] 10.222.0.99 () {58 vars in 1282 bytes} [Thu Jul 14 15:23:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 15:23:23.425597 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4bcbba-3fb6-4a2a-8508-e1a9e4bb5911 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:23.428488 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dca1b36-97e0-4113-84a0-1b2e18698961 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:23.523089 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:23.560206 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4bcbba-3fb6-4a2a-8508-e1a9e4bb5911 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:23.614686 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dca1b36-97e0-4113-84a0-1b2e18698961 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:23.632916 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551c7ad7-dc37-4356-ba65-d40837c91e7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:23.633519 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551c7ad7-dc37-4356-ba65-d40837c91e7a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:23.633807 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551c7ad7-dc37-4356-ba65-d40837c91e7a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:23.634087 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551c7ad7-dc37-4356-ba65-d40837c91e7a None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:23.635076 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551c7ad7-dc37-4356-ba65-d40837c91e7a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:23.635393 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551c7ad7-dc37-4356-ba65-d40837c91e7a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:23.636800 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551c7ad7-dc37-4356-ba65-d40837c91e7a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:23.660867 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 353/720] 10.222.0.99 () {56 vars in 1109 bytes} [Thu Jul 14 15:23:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:23:23.667718 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551c7ad7-dc37-4356-ba65-d40837c91e7a None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:23.693388 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b3b94f-2be5-4b38-8deb-53390ce92ce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:23.694279 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b3b94f-2be5-4b38-8deb-53390ce92ce5 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:23.694568 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b3b94f-2be5-4b38-8deb-53390ce92ce5 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:23.694849 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b3b94f-2be5-4b38-8deb-53390ce92ce5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:23.720042 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b3b94f-2be5-4b38-8deb-53390ce92ce5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:23.722231 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b3b94f-2be5-4b38-8deb-53390ce92ce5 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:23.751982 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:23.765703 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-85b3b94f-2be5-4b38-8deb-53390ce92ce5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:23:23.765703 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:23.765703 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:23.765703 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:23.765703 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:23.765703 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:23.765703 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:23.765703 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:23.765703 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:23.765703 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:23.765703 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:23.765703 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:23.765703 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:23:23.765703 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:23:23.765703 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:23:23.766658 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:23:23.766658 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:23.766658 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:23.766658 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:23.766658 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:23.766658 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:23.766658 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:23.766658 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:23.766658 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:23.766658 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:23.766658 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:23.766658 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:23.766658 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:23.766658 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:23.766658 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:23.766658 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:23.766658 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:23.767646 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:23:23.767646 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:23:23.767646 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:23:23.767646 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:23:23.767646 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 368/721] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:23:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 15:23:23.776598 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b3b94f-2be5-4b38-8deb-53390ce92ce5 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:23.807397 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551c7ad7-dc37-4356-ba65-d40837c91e7a None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:23.847067 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3267532-68d5-4ac0-8013-b45aac059f94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:23.848033 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3267532-68d5-4ac0-8013-b45aac059f94 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:23.848327 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3267532-68d5-4ac0-8013-b45aac059f94 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:23.848725 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3267532-68d5-4ac0-8013-b45aac059f94 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:23.857228 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3267532-68d5-4ac0-8013-b45aac059f94 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:23.858971 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3267532-68d5-4ac0-8013-b45aac059f94 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:23.876030 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-e3267532-68d5-4ac0-8013-b45aac059f94 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 15:23:23.876030 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:23.876030 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:23.876030 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:23.876030 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:23.876030 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:23.876030 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:23.876030 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:23.876030 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:23.876030 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:23.876030 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:23.876030 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:23.876030 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 15:23:23.876030 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 15:23:23.876030 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 15:23:23.876030 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 15:23:23.877081 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:23.877081 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:23.877081 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 15:23:23.877081 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:23:23.877081 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 15:23:23.877081 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:23:23.877081 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:23.877081 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:23.877081 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:23.877081 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:23.877081 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:23.877081 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:23.877081 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:23.877081 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:23.877081 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:23.877081 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:23.877081 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:23.877081 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:23.878024 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:23.878024 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:23.878024 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 15:23:23.878024 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 15:23:23.878024 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 15:23:23.878024 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 15:23:23.878024 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 15:23:23.878024 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 15:23:23.878024 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 15:23:23.878024 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:23:23.878024 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 354/722] 10.222.0.99 () {56 vars in 1117 bytes} [Thu Jul 14 15:23:23 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:23.883480 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3267532-68d5-4ac0-8013-b45aac059f94 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:23.974854 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b3b94f-2be5-4b38-8deb-53390ce92ce5 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:23.982452 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:24.024815 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3267532-68d5-4ac0-8013-b45aac059f94 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:24.036152 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403341ce-85d3-4c71-a70c-19aa49299f03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:24.036832 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403341ce-85d3-4c71-a70c-19aa49299f03 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:24.037117 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403341ce-85d3-4c71-a70c-19aa49299f03 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:24.037393 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403341ce-85d3-4c71-a70c-19aa49299f03 None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:24.038130 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403341ce-85d3-4c71-a70c-19aa49299f03 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:24.038435 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403341ce-85d3-4c71-a70c-19aa49299f03 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:24.039891 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403341ce-85d3-4c71-a70c-19aa49299f03 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:24.053287 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 369/723] 10.222.0.99 () {56 vars in 1109 bytes} [Thu Jul 14 15:23:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:23:24.054635 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6e0246-5f89-4a44-8068-3a8622803aa3 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:24.054927 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6e0246-5f89-4a44-8068-3a8622803aa3 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:24.055202 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6e0246-5f89-4a44-8068-3a8622803aa3 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:24.059899 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 370/724] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:23:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:23:24.060735 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6e0246-5f89-4a44-8068-3a8622803aa3 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:24.063990 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:24.074353 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5835aff-e9cd-431d-86ca-16980314e107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:24.075895 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5835aff-e9cd-431d-86ca-16980314e107 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:24.076656 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5835aff-e9cd-431d-86ca-16980314e107 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:24.078512 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5835aff-e9cd-431d-86ca-16980314e107 None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:24.080296 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5835aff-e9cd-431d-86ca-16980314e107 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:24.080870 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5835aff-e9cd-431d-86ca-16980314e107 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:24.083079 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5835aff-e9cd-431d-86ca-16980314e107 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:24.107268 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 355/725] 10.222.0.99 () {56 vars in 1149 bytes} [Thu Jul 14 15:23:23 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:23:24.109789 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982bedee-2300-453a-b0dc-87d6c1ff8e90 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:24.109789 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982bedee-2300-453a-b0dc-87d6c1ff8e90 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:24.109973 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982bedee-2300-453a-b0dc-87d6c1ff8e90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:24.133078 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-982bedee-2300-453a-b0dc-87d6c1ff8e90 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:23:24.134837 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-982bedee-2300-453a-b0dc-87d6c1ff8e90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:23:24.228601 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6e0246-5f89-4a44-8068-3a8622803aa3 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:24.239892 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 356/726] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:23:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:23:24.240961 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982bedee-2300-453a-b0dc-87d6c1ff8e90 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:24.294701 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99072b7c-80fe-481b-92da-b9c0dfcf4f64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:24.295617 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99072b7c-80fe-481b-92da-b9c0dfcf4f64 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:24.296446 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99072b7c-80fe-481b-92da-b9c0dfcf4f64 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:24.296446 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99072b7c-80fe-481b-92da-b9c0dfcf4f64 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:24.304840 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99072b7c-80fe-481b-92da-b9c0dfcf4f64 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:24.306304 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99072b7c-80fe-481b-92da-b9c0dfcf4f64 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:24.322084 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-99072b7c-80fe-481b-92da-b9c0dfcf4f64 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:23:24.322084 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:24.322084 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:24.322084 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:24.322084 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:24.322084 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:24.322084 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:24.322084 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:24.322084 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:24.322084 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:24.322084 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:24.322084 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:24.322084 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 15:23:24.322084 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 15:23:24.322084 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 15:23:24.323103 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 15:23:24.323103 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:24.323103 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:24.323103 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:24.323103 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:24.323103 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:24.323103 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:24.323103 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:24.323103 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:24.323103 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:24.323103 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:24.323103 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:24.323103 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:24.323103 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:24.323103 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:24.323103 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:24.323103 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:24.324653 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 15:23:24.324653 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 15:23:24.324653 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 15:23:24.324653 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 15:23:24.324653 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 15:23:24.324653 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 15:23:24.324653 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:23:24.324653 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:23:24.324653 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 371/727] 10.222.0.99 () {56 vars in 1111 bytes} [Thu Jul 14 15:23:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:24.324653 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99072b7c-80fe-481b-92da-b9c0dfcf4f64 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:24.360772 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:24.385662 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:24.392412 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982bedee-2300-453a-b0dc-87d6c1ff8e90 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:24.433665 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985b0dc8-b8bc-4e6e-8967-971d7b14d919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:24.434326 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985b0dc8-b8bc-4e6e-8967-971d7b14d919 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:24.434613 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985b0dc8-b8bc-4e6e-8967-971d7b14d919 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:24.434976 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985b0dc8-b8bc-4e6e-8967-971d7b14d919 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:24.449916 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99072b7c-80fe-481b-92da-b9c0dfcf4f64 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:24.495331 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd5ef09-59fa-45ad-9096-9d867cc636cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:24.496278 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd5ef09-59fa-45ad-9096-9d867cc636cb None admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:24.496278 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd5ef09-59fa-45ad-9096-9d867cc636cb None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:24.496578 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd5ef09-59fa-45ad-9096-9d867cc636cb None admin] PATH_INFO: `/v3/services` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:24.497346 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd5ef09-59fa-45ad-9096-9d867cc636cb None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:24.499437 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd5ef09-59fa-45ad-9096-9d867cc636cb None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:24.526089 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 372/728] 10.222.0.99 () {60 vars in 1141 bytes} [Thu Jul 14 15:23:24 2022] POST /identity/v3/services => generated 240 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:23:24.533100 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd5ef09-59fa-45ad-9096-9d867cc636cb None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:24.535370 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985b0dc8-b8bc-4e6e-8967-971d7b14d919 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:24.538173 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985b0dc8-b8bc-4e6e-8967-971d7b14d919 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:24.692016 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:24.696029 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:24.698288 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 357/729] 10.222.0.99 () {58 vars in 1282 bytes} [Thu Jul 14 15:23:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 15:23:24.700088 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560786ea-05d0-42bf-aef2-31653eb24bdb None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:24.700764 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560786ea-05d0-42bf-aef2-31653eb24bdb None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:24.701301 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560786ea-05d0-42bf-aef2-31653eb24bdb None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:24.707985 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 358/730] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:23:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:23:24.709509 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560786ea-05d0-42bf-aef2-31653eb24bdb None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:24.731311 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd5ef09-59fa-45ad-9096-9d867cc636cb None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:24.776526 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94830ca1-d8d7-4c7f-94dd-2239bbdacb83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:24.777205 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94830ca1-d8d7-4c7f-94dd-2239bbdacb83 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:24.777491 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94830ca1-d8d7-4c7f-94dd-2239bbdacb83 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:24.777767 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94830ca1-d8d7-4c7f-94dd-2239bbdacb83 None admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:24.778566 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94830ca1-d8d7-4c7f-94dd-2239bbdacb83 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:24.778874 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94830ca1-d8d7-4c7f-94dd-2239bbdacb83 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:24.780397 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94830ca1-d8d7-4c7f-94dd-2239bbdacb83 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:24.794296 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 373/731] 10.222.0.99 () {56 vars in 1148 bytes} [Thu Jul 14 15:23:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:24.801548 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d9e318-5398-40e4-85da-55ca60178002 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:24.801935 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d9e318-5398-40e4-85da-55ca60178002 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:24.802226 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d9e318-5398-40e4-85da-55ca60178002 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:24.820159 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-c7d9e318-5398-40e4-85da-55ca60178002 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:23:24.821295 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-c7d9e318-5398-40e4-85da-55ca60178002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:23:24.865084 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560786ea-05d0-42bf-aef2-31653eb24bdb None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:24.909933 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 374/732] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:23:24 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:23:24.917664 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d9e318-5398-40e4-85da-55ca60178002 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:24.922524 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f62dd3-33ae-4801-a818-40d093c2a83f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:24.923332 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f62dd3-33ae-4801-a818-40d093c2a83f None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:24.923631 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f62dd3-33ae-4801-a818-40d093c2a83f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:24.923931 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f62dd3-33ae-4801-a818-40d093c2a83f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:24.943176 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f62dd3-33ae-4801-a818-40d093c2a83f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:24.944686 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f62dd3-33ae-4801-a818-40d093c2a83f None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:24.963920 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-63f62dd3-33ae-4801-a818-40d093c2a83f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:23:24.963920 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:24.963920 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:24.963920 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:24.963920 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:24.963920 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:24.963920 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:24.963920 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:24.963920 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:24.963920 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:24.963920 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:24.963920 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:24.963920 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:23:24.963920 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:23:24.963920 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:23:24.964884 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:23:24.964884 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:24.964884 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:24.964884 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:24.964884 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:24.964884 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:24.964884 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:24.964884 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:24.964884 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:24.964884 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:24.964884 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:24.964884 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:24.964884 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:24.964884 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:24.964884 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:24.964884 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:24.964884 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:24.965775 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:23:24.965775 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:23:24.965775 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:23:24.965775 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:23:24.965775 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 359/733] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:23:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:24.966480 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f62dd3-33ae-4801-a818-40d093c2a83f None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:25.017329 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:25.028203 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:25.052882 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d9e318-5398-40e4-85da-55ca60178002 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:25.083534 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f62dd3-33ae-4801-a818-40d093c2a83f None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:25.092143 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634a8c14-2c0b-484e-8fad-8fe94df947b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:25.093889 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634a8c14-2c0b-484e-8fad-8fe94df947b6 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:25.094276 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634a8c14-2c0b-484e-8fad-8fe94df947b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:25.094651 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634a8c14-2c0b-484e-8fad-8fe94df947b6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:25.095629 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634a8c14-2c0b-484e-8fad-8fe94df947b6 None admin] RBAC: Adding query filter params (role.id=d0f6ba1314f14d9f9c816bdb5d0c3aac, scope.project.id=9048d1a6f00443ec92f11471babe5a56, user.id=3a9b1c2622934c5185f3b5f1ea623bc0) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:25.096044 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634a8c14-2c0b-484e-8fad-8fe94df947b6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:25.097579 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634a8c14-2c0b-484e-8fad-8fe94df947b6 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:25.108975 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 375/734] 10.222.0.99 () {56 vars in 1374 bytes} [Thu Jul 14 15:23:24 2022] GET /identity/v3/role_assignments?user.id=3a9b1c2622934c5185f3b5f1ea623bc0&scope.project.id=9048d1a6f00443ec92f11471babe5a56&role.id=d0f6ba1314f14d9f9c816bdb5d0c3aac => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:25.110343 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634a8c14-2c0b-484e-8fad-8fe94df947b6 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:25.143820 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6431ac-579e-4608-902f-e6237731ec78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:25.144709 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6431ac-579e-4608-902f-e6237731ec78 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:25.145000 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6431ac-579e-4608-902f-e6237731ec78 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:25.145341 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6431ac-579e-4608-902f-e6237731ec78 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:25.170176 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6431ac-579e-4608-902f-e6237731ec78 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:23:25.187485 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-ce6431ac-579e-4608-902f-e6237731ec78 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:23:25.187485 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:25.187485 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:25.187485 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:25.187485 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:25.187485 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:25.187485 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:25.187485 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:25.187485 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:25.187485 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:25.187485 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:25.187485 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:25.187485 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:23:25.187485 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:23:25.187485 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:23:25.187485 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:23:25.188859 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:23:25.188859 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:23:25.188859 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:23:25.188859 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:23:25.188859 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:25.188859 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:25.188859 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:25.188859 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:25.188859 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:25.188859 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:25.188859 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:25.188859 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:25.188859 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:25.188859 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:25.188859 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:25.188859 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:25.189885 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:25.189885 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:25.189885 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:25.189885 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:25.189885 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:23:25.189885 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:23:25.189885 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:23:25.189885 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:23:25.189885 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:23:25.189885 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:23:25.189885 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:23:25.189885 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:23:25.194756 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 360/735] 10.222.0.99 () {56 vars in 1096 bytes} [Thu Jul 14 15:23:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:25.196028 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6431ac-579e-4608-902f-e6237731ec78 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:25.257235 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634a8c14-2c0b-484e-8fad-8fe94df947b6 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:25.303773 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe032b33-f232-4a66-95c2-884525a7b940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:25.304703 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe032b33-f232-4a66-95c2-884525a7b940 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:25.305140 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe032b33-f232-4a66-95c2-884525a7b940 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:25.305594 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe032b33-f232-4a66-95c2-884525a7b940 None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:25.306482 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe032b33-f232-4a66-95c2-884525a7b940 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:25.306938 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe032b33-f232-4a66-95c2-884525a7b940 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:25.308525 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe032b33-f232-4a66-95c2-884525a7b940 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:25.314224 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:25.334347 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 376/736] 10.222.0.99 () {56 vars in 1109 bytes} [Thu Jul 14 15:23:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:25.341891 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe032b33-f232-4a66-95c2-884525a7b940 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:25.401774 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6431ac-579e-4608-902f-e6237731ec78 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:25.422357 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:25.478321 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfde6d88-dc9d-418b-9f2f-7afaa31a0f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:25.479362 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfde6d88-dc9d-418b-9f2f-7afaa31a0f37 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:25.479898 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfde6d88-dc9d-418b-9f2f-7afaa31a0f37 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:25.480473 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfde6d88-dc9d-418b-9f2f-7afaa31a0f37 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:25.482455 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfde6d88-dc9d-418b-9f2f-7afaa31a0f37 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:25.483191 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfde6d88-dc9d-418b-9f2f-7afaa31a0f37 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:25.486276 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfde6d88-dc9d-418b-9f2f-7afaa31a0f37 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:25.497067 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe032b33-f232-4a66-95c2-884525a7b940 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:25.510605 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 361/737] 10.222.0.99 () {56 vars in 1099 bytes} [Thu Jul 14 15:23:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:23:25.512186 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfde6d88-dc9d-418b-9f2f-7afaa31a0f37 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:25.574415 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544a4dfa-3e16-423b-b5b3-45016494bbb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:25.578132 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544a4dfa-3e16-423b-b5b3-45016494bbb2 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:25.580245 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544a4dfa-3e16-423b-b5b3-45016494bbb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:25.580750 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544a4dfa-3e16-423b-b5b3-45016494bbb2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:25.603644 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544a4dfa-3e16-423b-b5b3-45016494bbb2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:25.605460 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544a4dfa-3e16-423b-b5b3-45016494bbb2 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:25.628184 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-544a4dfa-3e16-423b-b5b3-45016494bbb2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:23:25.628184 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:25.628184 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:25.628184 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:25.628184 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:25.628184 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:25.628184 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:25.628184 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:25.628184 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:25.628184 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:25.628184 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:25.628184 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:25.628184 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 15:23:25.628184 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 15:23:25.628184 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 15:23:25.629139 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 15:23:25.629139 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:25.629139 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:25.629139 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:25.629139 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:25.629139 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:25.629139 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:25.629139 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:25.629139 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:25.629139 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:25.629139 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:25.629139 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:25.629139 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:25.629139 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:25.629139 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:25.629139 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:25.629139 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:25.630046 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 15:23:25.630046 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 15:23:25.630046 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 15:23:25.630046 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 15:23:25.630046 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 15:23:25.630046 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 15:23:25.630046 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:23:25.630046 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:23:25.632649 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 377/738] 10.222.0.99 () {56 vars in 1111 bytes} [Thu Jul 14 15:23:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 15:23:25.639026 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544a4dfa-3e16-423b-b5b3-45016494bbb2 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:25.704178 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfde6d88-dc9d-418b-9f2f-7afaa31a0f37 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:25.727986 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:25.747176 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cedd2b-b4a0-4a05-8795-2e1f561c97fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:25.748331 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cedd2b-b4a0-4a05-8795-2e1f561c97fe None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:25.748832 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cedd2b-b4a0-4a05-8795-2e1f561c97fe None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:25.749772 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cedd2b-b4a0-4a05-8795-2e1f561c97fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:25.762144 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cedd2b-b4a0-4a05-8795-2e1f561c97fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:25.764010 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cedd2b-b4a0-4a05-8795-2e1f561c97fe None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:25.781808 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544a4dfa-3e16-423b-b5b3-45016494bbb2 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:25.784591 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-84cedd2b-b4a0-4a05-8795-2e1f561c97fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:23:25.784591 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:25.784591 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:25.784591 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:25.784591 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:25.784591 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:25.784591 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:25.784591 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:25.784591 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:25.784591 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:25.784591 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:25.784591 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:25.784591 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:23:25.784591 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:23:25.784591 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:23:25.785480 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:23:25.785480 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:25.785480 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:25.785480 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:25.785480 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:25.785480 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:25.785480 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:25.785480 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:25.785480 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:25.785480 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:25.785480 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:25.785480 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:25.785480 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:25.785480 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:25.785480 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:25.785480 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:25.785480 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:25.786365 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:23:25.786365 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:23:25.786365 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:23:25.786365 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:23:25.788835 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 362/739] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:23:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:25.790428 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a70fa18-8b83-41b0-bc0c-fd5f4dc635a6 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:25.790931 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a70fa18-8b83-41b0-bc0c-fd5f4dc635a6 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:25.791434 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a70fa18-8b83-41b0-bc0c-fd5f4dc635a6 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:25.798258 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 363/740] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:23:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:23:25.799447 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a70fa18-8b83-41b0-bc0c-fd5f4dc635a6 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:25.822325 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09f20c7-eea2-45e4-9612-e7e21ee9e708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:25.823350 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09f20c7-eea2-45e4-9612-e7e21ee9e708 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:25.823913 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09f20c7-eea2-45e4-9612-e7e21ee9e708 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:25.824403 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09f20c7-eea2-45e4-9612-e7e21ee9e708 None admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:25.825447 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09f20c7-eea2-45e4-9612-e7e21ee9e708 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:25.825994 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09f20c7-eea2-45e4-9612-e7e21ee9e708 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:25.827783 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09f20c7-eea2-45e4-9612-e7e21ee9e708 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:25.832039 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:25.841341 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 378/741] 10.222.0.99 () {56 vars in 1148 bytes} [Thu Jul 14 15:23:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:23:25.843566 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60d8758-fbc8-4e20-8efe-fc9b412766d8 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:25.844134 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60d8758-fbc8-4e20-8efe-fc9b412766d8 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:25.844621 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60d8758-fbc8-4e20-8efe-fc9b412766d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:25.864147 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-d60d8758-fbc8-4e20-8efe-fc9b412766d8 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:23:25.865735 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-d60d8758-fbc8-4e20-8efe-fc9b412766d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:23:25.933280 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a70fa18-8b83-41b0-bc0c-fd5f4dc635a6 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:25.954399 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 379/742] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:23:25 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:23:25.955212 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60d8758-fbc8-4e20-8efe-fc9b412766d8 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:25.977185 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa49437-b331-4625-8144-c0bac1511f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:25.978276 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa49437-b331-4625-8144-c0bac1511f0b None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:25.978848 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa49437-b331-4625-8144-c0bac1511f0b None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:25.979174 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa49437-b331-4625-8144-c0bac1511f0b None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:25.983769 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa49437-b331-4625-8144-c0bac1511f0b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:25.983769 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa49437-b331-4625-8144-c0bac1511f0b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:25.983769 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa49437-b331-4625-8144-c0bac1511f0b None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:25.997862 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 364/743] 10.222.0.99 () {56 vars in 1109 bytes} [Thu Jul 14 15:23:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:25.998774 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa49437-b331-4625-8144-c0bac1511f0b None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:26.033412 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:26.106610 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60d8758-fbc8-4e20-8efe-fc9b412766d8 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:26.125665 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:26.148940 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa49437-b331-4625-8144-c0bac1511f0b None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:26.160540 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a626e501-0447-4216-ad8a-96451458821b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:26.161681 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a626e501-0447-4216-ad8a-96451458821b None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:26.161979 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a626e501-0447-4216-ad8a-96451458821b None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:26.162268 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a626e501-0447-4216-ad8a-96451458821b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:26.163526 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a626e501-0447-4216-ad8a-96451458821b None admin] RBAC: Adding query filter params (role.id=74e3b597938f49d095cb424c54b5a811, scope.project.id=9048d1a6f00443ec92f11471babe5a56, user.id=adaeadf613954d7393b3fd3bae68c362) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:26.163899 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a626e501-0447-4216-ad8a-96451458821b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:26.165667 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a626e501-0447-4216-ad8a-96451458821b None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:26.176774 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 380/744] 10.222.0.99 () {56 vars in 1374 bytes} [Thu Jul 14 15:23:25 2022] GET /identity/v3/role_assignments?user.id=adaeadf613954d7393b3fd3bae68c362&scope.project.id=9048d1a6f00443ec92f11471babe5a56&role.id=74e3b597938f49d095cb424c54b5a811 => generated 259 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:26.178011 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a626e501-0447-4216-ad8a-96451458821b None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:26.195221 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b7b77d-2f4a-4a13-a681-d42625bf1982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:26.198796 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b7b77d-2f4a-4a13-a681-d42625bf1982 None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:26.201200 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b7b77d-2f4a-4a13-a681-d42625bf1982 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:26.202811 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b7b77d-2f4a-4a13-a681-d42625bf1982 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:26.204517 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b7b77d-2f4a-4a13-a681-d42625bf1982 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:26.207550 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b7b77d-2f4a-4a13-a681-d42625bf1982 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:26.232765 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 365/745] 10.222.0.99 () {60 vars in 1141 bytes} [Thu Jul 14 15:23:25 2022] POST /identity/v3/services => generated 251 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:23:26.335283 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a626e501-0447-4216-ad8a-96451458821b None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:26.350263 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:26.391587 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9610bc-df02-4b94-8234-e6030bca9d80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:26.392443 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9610bc-df02-4b94-8234-e6030bca9d80 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:26.392728 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9610bc-df02-4b94-8234-e6030bca9d80 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:26.393007 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9610bc-df02-4b94-8234-e6030bca9d80 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:26.402846 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9610bc-df02-4b94-8234-e6030bca9d80 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:26.404540 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9610bc-df02-4b94-8234-e6030bca9d80 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:26.423221 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-3f9610bc-df02-4b94-8234-e6030bca9d80 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 14 15:23:26.423221 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:26.423221 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:26.423221 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:26.423221 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:26.423221 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:26.423221 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:26.423221 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:26.423221 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:26.423221 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:26.423221 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:26.423221 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:26.423221 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 15:23:26.423221 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 15:23:26.423221 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 15:23:26.423221 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 15:23:26.424280 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:26.424280 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:26.424280 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 15:23:26.424280 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:23:26.424280 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 15:23:26.424280 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:23:26.424280 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:26.424280 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:26.424280 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:26.424280 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:26.424280 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:26.424280 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:26.424280 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:26.424280 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:26.424280 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:26.424280 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:26.424280 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:26.424280 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:26.425204 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:26.425204 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:26.425204 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 15:23:26.425204 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 15:23:26.425204 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 15:23:26.425204 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 15:23:26.425204 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 15:23:26.425204 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 15:23:26.425204 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 14 15:23:26.425204 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:23:26.425204 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 381/746] 10.222.0.99 () {56 vars in 1096 bytes} [Thu Jul 14 15:23:26 2022] GET /identity/v3/users/swift => generated 83 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:26.431551 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b7b77d-2f4a-4a13-a681-d42625bf1982 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:26.550453 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b7b77d-2f4a-4a13-a681-d42625bf1982 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:26.589522 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff75949f-6d79-474e-9223-f0a9dd16d4ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:26.590133 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff75949f-6d79-474e-9223-f0a9dd16d4ce None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:26.590420 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff75949f-6d79-474e-9223-f0a9dd16d4ce None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:26.590701 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff75949f-6d79-474e-9223-f0a9dd16d4ce None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:26.591563 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff75949f-6d79-474e-9223-f0a9dd16d4ce None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:26.591921 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff75949f-6d79-474e-9223-f0a9dd16d4ce None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:26.594101 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff75949f-6d79-474e-9223-f0a9dd16d4ce None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:26.614483 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 366/747] 10.222.0.99 () {56 vars in 1135 bytes} [Thu Jul 14 15:23:26 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:26.620127 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9610bc-df02-4b94-8234-e6030bca9d80 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:26.737663 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9610bc-df02-4b94-8234-e6030bca9d80 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:26.776853 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95707f6b-37b6-4dc3-b9fe-fa15b9242c6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:26.777515 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95707f6b-37b6-4dc3-b9fe-fa15b9242c6c None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:26.777800 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95707f6b-37b6-4dc3-b9fe-fa15b9242c6c None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:26.778080 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95707f6b-37b6-4dc3-b9fe-fa15b9242c6c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:26.815457 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:26.854913 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95707f6b-37b6-4dc3-b9fe-fa15b9242c6c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:26.856337 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95707f6b-37b6-4dc3-b9fe-fa15b9242c6c None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:26.964639 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 382/748] 10.222.0.99 () {58 vars in 1282 bytes} [Thu Jul 14 15:23:26 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 15:23:26.971068 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff75949f-6d79-474e-9223-f0a9dd16d4ce None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:26.974458 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:27.098828 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff75949f-6d79-474e-9223-f0a9dd16d4ce None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:27.144963 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9bad01-154f-4364-9c39-55e21cced427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:27.145783 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9bad01-154f-4364-9c39-55e21cced427 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:27.146078 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9bad01-154f-4364-9c39-55e21cced427 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:27.146366 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9bad01-154f-4364-9c39-55e21cced427 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:27.158846 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9bad01-154f-4364-9c39-55e21cced427 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:27.160352 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9bad01-154f-4364-9c39-55e21cced427 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:27.192209 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-ec9bad01-154f-4364-9c39-55e21cced427 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:23:27.192209 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:27.192209 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:27.192209 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:27.192209 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:27.192209 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:27.192209 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:27.192209 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:27.192209 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:27.192209 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:27.192209 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:27.192209 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:27.192209 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:23:27.192209 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:23:27.192209 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:23:27.193198 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:23:27.193198 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:27.193198 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:27.193198 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:27.193198 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:27.193198 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:27.193198 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:27.193198 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:27.193198 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:27.193198 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:27.193198 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:27.193198 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:27.193198 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:27.193198 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:27.193198 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:27.193198 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:27.193198 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:27.194084 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:23:27.194084 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:23:27.194084 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:23:27.194084 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:23:27.194084 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 367/749] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:23:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:27.199377 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95707f6b-37b6-4dc3-b9fe-fa15b9242c6c None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:27.228881 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a17d3a-04e6-4d43-a31a-b028cc934e1a None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:27.229515 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a17d3a-04e6-4d43-a31a-b028cc934e1a None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:27.230041 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a17d3a-04e6-4d43-a31a-b028cc934e1a None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:27.231966 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 368/750] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:23:27 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:23:27.238363 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9153815d-ca2b-496d-83e6-8a197e5b7596 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:27.239039 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9153815d-ca2b-496d-83e6-8a197e5b7596 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:27.239528 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9153815d-ca2b-496d-83e6-8a197e5b7596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:27.259355 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-9153815d-ca2b-496d-83e6-8a197e5b7596 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:23:27.261173 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-9153815d-ca2b-496d-83e6-8a197e5b7596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:23:27.335943 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:27.349117 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:27.395640 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95707f6b-37b6-4dc3-b9fe-fa15b9242c6c None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:27.397857 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 369/751] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:23:27 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 15:23:27.449934 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f36cda-6549-4929-8e11-29cebd519141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:27.450624 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f36cda-6549-4929-8e11-29cebd519141 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:27.450920 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f36cda-6549-4929-8e11-29cebd519141 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:27.451202 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f36cda-6549-4929-8e11-29cebd519141 None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:27.458966 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f36cda-6549-4929-8e11-29cebd519141 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:27.459299 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f36cda-6549-4929-8e11-29cebd519141 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:27.459623 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9153815d-ca2b-496d-83e6-8a197e5b7596 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:27.460771 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f36cda-6549-4929-8e11-29cebd519141 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:27.474362 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 383/752] 10.222.0.99 () {56 vars in 1109 bytes} [Thu Jul 14 15:23:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:27.479629 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f36cda-6549-4929-8e11-29cebd519141 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:27.596278 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f36cda-6549-4929-8e11-29cebd519141 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:27.627286 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:27.637632 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cc8a53-3c82-4d54-90e5-3bf835f254cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:27.638453 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cc8a53-3c82-4d54-90e5-3bf835f254cf None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:27.638741 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cc8a53-3c82-4d54-90e5-3bf835f254cf None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:27.639021 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cc8a53-3c82-4d54-90e5-3bf835f254cf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:27.647371 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cc8a53-3c82-4d54-90e5-3bf835f254cf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:27.648807 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cc8a53-3c82-4d54-90e5-3bf835f254cf None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:27.665434 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-60cc8a53-3c82-4d54-90e5-3bf835f254cf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:23:27.665434 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:27.665434 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:27.665434 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:27.665434 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:27.665434 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:27.665434 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:27.665434 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:27.665434 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:27.665434 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:27.665434 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:27.665434 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:27.665434 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 15:23:27.665434 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 15:23:27.665434 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 15:23:27.666390 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 15:23:27.666390 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:27.666390 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:27.666390 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:27.666390 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:27.666390 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:27.666390 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:27.666390 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:27.666390 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:27.666390 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:27.666390 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:27.666390 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:27.666390 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:27.666390 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:27.666390 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:27.666390 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:27.666390 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:27.667933 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 15:23:27.667933 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 15:23:27.667933 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 15:23:27.667933 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 15:23:27.667933 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 15:23:27.667933 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 15:23:27.667933 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:23:27.667933 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:23:27.667933 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 384/753] 10.222.0.99 () {56 vars in 1111 bytes} [Thu Jul 14 15:23:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 15:23:27.668433 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ed1114-de42-4d25-8308-f1b86ea70bf4 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:27.668673 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ed1114-de42-4d25-8308-f1b86ea70bf4 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:27.668875 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ed1114-de42-4d25-8308-f1b86ea70bf4 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:27.674183 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 385/754] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:23:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:23:27.674468 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ed1114-de42-4d25-8308-f1b86ea70bf4 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:27.688852 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9153815d-ca2b-496d-83e6-8a197e5b7596 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:27.732560 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1c73fe-8d27-489d-be80-0aeeda2a1a6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:27.733513 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1c73fe-8d27-489d-be80-0aeeda2a1a6d None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:27.733876 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1c73fe-8d27-489d-be80-0aeeda2a1a6d None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:27.734238 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1c73fe-8d27-489d-be80-0aeeda2a1a6d None admin] PATH_INFO: `/v3/services/network` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:27.735015 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1c73fe-8d27-489d-be80-0aeeda2a1a6d None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:27.736478 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1c73fe-8d27-489d-be80-0aeeda2a1a6d None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:27.749122 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-5a1c73fe-8d27-489d-be80-0aeeda2a1a6d None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 14 15:23:27.749122 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:27.749122 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 15:23:27.749122 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 15:23:27.749122 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 15:23:27.749122 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 15:23:27.749122 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 15:23:27.749122 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 15:23:27.749122 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 14 15:23:27.749122 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:23:27.749122 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 15:23:27.749122 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:23:27.749122 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:27.749122 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:27.749122 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:27.749122 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:27.749122 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:27.750561 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:27.750561 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:27.750561 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:27.750561 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:27.750561 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:27.750561 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:27.750561 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 15:23:27.750561 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 15:23:27.750561 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 15:23:27.750561 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 15:23:27.750561 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:27.750561 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:27.750561 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:27.750561 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:27.750561 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:27.750561 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:27.751972 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:27.751972 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:27.751972 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:27.751972 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:27.751972 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:27.751972 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:27.751972 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:27.751972 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:27.751972 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:27.751972 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:27.751972 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 15:23:27.751972 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 15:23:27.751972 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 14 15:23:27.751972 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:23:27.751972 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 370/755] 10.222.0.99 () {56 vars in 1111 bytes} [Thu Jul 14 15:23:27 2022] GET /identity/v3/services/network => generated 88 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:27.753054 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d01db2-11f7-4617-b752-b7b71caa4b00 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:27.753054 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d01db2-11f7-4617-b752-b7b71caa4b00 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:27.753189 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d01db2-11f7-4617-b752-b7b71caa4b00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:27.777170 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-b8d01db2-11f7-4617-b752-b7b71caa4b00 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:23:27.778701 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b8d01db2-11f7-4617-b752-b7b71caa4b00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:23:27.808314 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ed1114-de42-4d25-8308-f1b86ea70bf4 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:27.839789 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:27.853815 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88433cee-bae3-4029-9c33-8769019c221b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:27.854594 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88433cee-bae3-4029-9c33-8769019c221b None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:27.854887 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88433cee-bae3-4029-9c33-8769019c221b None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:27.855271 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88433cee-bae3-4029-9c33-8769019c221b None admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:27.856115 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88433cee-bae3-4029-9c33-8769019c221b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:27.856426 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88433cee-bae3-4029-9c33-8769019c221b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:27.857981 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88433cee-bae3-4029-9c33-8769019c221b None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:27.867166 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 371/756] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:23:27 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 15:23:27.868087 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d01db2-11f7-4617-b752-b7b71caa4b00 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:27.872200 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 386/757] 10.222.0.99 () {56 vars in 1148 bytes} [Thu Jul 14 15:23:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:27.881753 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88433cee-bae3-4029-9c33-8769019c221b None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:27.947216 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:27.997227 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d01db2-11f7-4617-b752-b7b71caa4b00 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:28.024742 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88433cee-bae3-4029-9c33-8769019c221b None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:28.037718 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66609b8f-d06b-4fa4-b5c3-ba1ccfeac6a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:28.038473 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66609b8f-d06b-4fa4-b5c3-ba1ccfeac6a7 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:28.038835 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66609b8f-d06b-4fa4-b5c3-ba1ccfeac6a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:28.039189 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66609b8f-d06b-4fa4-b5c3-ba1ccfeac6a7 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:28.040014 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66609b8f-d06b-4fa4-b5c3-ba1ccfeac6a7 None admin] RBAC: Adding query filter params (name=network) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:28.040405 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66609b8f-d06b-4fa4-b5c3-ba1ccfeac6a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:28.041856 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66609b8f-d06b-4fa4-b5c3-ba1ccfeac6a7 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:28.047337 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 372/758] 10.222.0.99 () {56 vars in 1112 bytes} [Thu Jul 14 15:23:27 2022] GET /identity/v3/services?name=network => generated 124 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:28.048730 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66609b8f-d06b-4fa4-b5c3-ba1ccfeac6a7 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:28.069357 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0392d0-ddb9-4f15-99fc-a4aaf9150964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:28.070267 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0392d0-ddb9-4f15-99fc-a4aaf9150964 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:28.070722 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0392d0-ddb9-4f15-99fc-a4aaf9150964 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:28.071128 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0392d0-ddb9-4f15-99fc-a4aaf9150964 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:28.072188 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0392d0-ddb9-4f15-99fc-a4aaf9150964 None admin] RBAC: Adding query filter params (role.id=d0f6ba1314f14d9f9c816bdb5d0c3aac, scope.project.id=9048d1a6f00443ec92f11471babe5a56, user.id=ddd2ec3b3ebc4aacb322c95270374451) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:28.072581 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0392d0-ddb9-4f15-99fc-a4aaf9150964 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:28.074173 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0392d0-ddb9-4f15-99fc-a4aaf9150964 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:28.087625 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 387/759] 10.222.0.99 () {56 vars in 1374 bytes} [Thu Jul 14 15:23:27 2022] GET /identity/v3/role_assignments?user.id=ddd2ec3b3ebc4aacb322c95270374451&scope.project.id=9048d1a6f00443ec92f11471babe5a56&role.id=d0f6ba1314f14d9f9c816bdb5d0c3aac => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:28.087625 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0392d0-ddb9-4f15-99fc-a4aaf9150964 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:28.126697 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:28.231569 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0392d0-ddb9-4f15-99fc-a4aaf9150964 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:28.244605 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:28.277028 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66609b8f-d06b-4fa4-b5c3-ba1ccfeac6a7 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:28.281149 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99ef99e-2020-498b-9ced-c567f35d41de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:28.282200 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99ef99e-2020-498b-9ced-c567f35d41de None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:28.282676 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99ef99e-2020-498b-9ced-c567f35d41de None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:28.284089 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99ef99e-2020-498b-9ced-c567f35d41de None admin] PATH_INFO: `/v3/services` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:28.285025 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99ef99e-2020-498b-9ced-c567f35d41de None admin] RBAC: Adding query filter params (type=network) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:28.288107 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99ef99e-2020-498b-9ced-c567f35d41de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:28.289632 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99ef99e-2020-498b-9ced-c567f35d41de None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:28.307174 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 388/760] 10.222.0.99 () {56 vars in 1112 bytes} [Thu Jul 14 15:23:28 2022] GET /identity/v3/services?type=network => generated 350 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:28.313020 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99ef99e-2020-498b-9ced-c567f35d41de None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:28.341444 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374ec972-d886-477e-a244-571317524375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:28.343362 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374ec972-d886-477e-a244-571317524375 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:28.343362 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374ec972-d886-477e-a244-571317524375 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:28.343651 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374ec972-d886-477e-a244-571317524375 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:28.374509 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374ec972-d886-477e-a244-571317524375 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:23:28.382671 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-374ec972-d886-477e-a244-571317524375 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:23:28.382671 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:28.382671 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:28.382671 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:28.382671 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:28.382671 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:28.382671 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:28.382671 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:28.382671 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:28.382671 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:28.382671 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:28.382671 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:28.382671 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:23:28.382671 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:23:28.382671 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:23:28.382671 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:23:28.383617 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:23:28.383617 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:23:28.383617 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:23:28.383617 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:23:28.383617 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:28.383617 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:28.383617 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:28.383617 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:28.383617 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:28.383617 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:28.383617 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:28.383617 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:28.383617 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:28.383617 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:28.383617 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:28.383617 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:28.384456 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:28.384456 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:28.384456 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:28.384456 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:28.384456 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:23:28.384456 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:23:28.384456 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:23:28.384456 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:23:28.384456 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:23:28.384456 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:23:28.384456 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:23:28.384456 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:23:28.398699 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 373/761] 10.222.0.99 () {56 vars in 1096 bytes} [Thu Jul 14 15:23:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:28.401355 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374ec972-d886-477e-a244-571317524375 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:28.463691 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99ef99e-2020-498b-9ced-c567f35d41de None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:28.553058 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d245a4-d291-4f1b-a1ac-893e26eb10d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:28.554195 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d245a4-d291-4f1b-a1ac-893e26eb10d5 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:28.554770 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d245a4-d291-4f1b-a1ac-893e26eb10d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:28.555164 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d245a4-d291-4f1b-a1ac-893e26eb10d5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:28.556342 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d245a4-d291-4f1b-a1ac-893e26eb10d5 None admin] RBAC: Adding query filter params (interface=public, service_id=1a856bcbd7c84d32a85d13d75f019bfc, region_id=RegionOne) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:28.556786 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d245a4-d291-4f1b-a1ac-893e26eb10d5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:28.558812 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d245a4-d291-4f1b-a1ac-893e26eb10d5 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:28.572476 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374ec972-d886-477e-a244-571317524375 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:28.576027 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:28.580214 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 389/762] 10.222.0.99 () {56 vars in 1251 bytes} [Thu Jul 14 15:23:28 2022] GET /identity/v3/endpoints?service_id=1a856bcbd7c84d32a85d13d75f019bfc&interface=public®ion_id=RegionOne => generated 194 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:23:28.587887 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d245a4-d291-4f1b-a1ac-893e26eb10d5 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:28.657506 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45f25d4-04d3-4187-a6bf-684012e712fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:28.658207 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45f25d4-04d3-4187-a6bf-684012e712fa None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:28.658484 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45f25d4-04d3-4187-a6bf-684012e712fa None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:28.658770 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45f25d4-04d3-4187-a6bf-684012e712fa None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:28.659544 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45f25d4-04d3-4187-a6bf-684012e712fa None admin] RBAC: Adding query filter params (name=admin) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:28.659886 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45f25d4-04d3-4187-a6bf-684012e712fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:28.661291 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45f25d4-04d3-4187-a6bf-684012e712fa None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:28.672261 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:28.682025 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 374/763] 10.222.0.99 () {56 vars in 1099 bytes} [Thu Jul 14 15:23:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:23:28.683223 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45f25d4-04d3-4187-a6bf-684012e712fa None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:28.737554 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d245a4-d291-4f1b-a1ac-893e26eb10d5 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:28.782728 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8999563e-dca0-4440-937c-707cee78a7fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:28.783989 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8999563e-dca0-4440-937c-707cee78a7fb None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:28.784497 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8999563e-dca0-4440-937c-707cee78a7fb None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:28.785013 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8999563e-dca0-4440-937c-707cee78a7fb None admin] PATH_INFO: `/v3/services` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:28.785900 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8999563e-dca0-4440-937c-707cee78a7fb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:28.787496 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8999563e-dca0-4440-937c-707cee78a7fb None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:28.794866 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 390/764] 10.222.0.99 () {56 vars in 1087 bytes} [Thu Jul 14 15:23:28 2022] GET /identity/v3/services => generated 772 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:28.797364 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3e1185-dbfe-40c0-b504-17aec56ed72c None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:28.797890 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3e1185-dbfe-40c0-b504-17aec56ed72c None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:28.798369 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3e1185-dbfe-40c0-b504-17aec56ed72c None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:28.800308 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 391/765] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:23:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:23:28.810082 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc24a198-5607-40b2-9892-e20afa4b3db6 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:28.810804 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc24a198-5607-40b2-9892-e20afa4b3db6 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:28.811466 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc24a198-5607-40b2-9892-e20afa4b3db6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:28.846763 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-dc24a198-5607-40b2-9892-e20afa4b3db6 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:23:28.852243 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-dc24a198-5607-40b2-9892-e20afa4b3db6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:23:28.885169 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45f25d4-04d3-4187-a6bf-684012e712fa None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:28.940857 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0bd2e1-84b6-4d9f-9f57-3ea7f9eceea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:28.942685 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0bd2e1-84b6-4d9f-9f57-3ea7f9eceea8 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:28.944003 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0bd2e1-84b6-4d9f-9f57-3ea7f9eceea8 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:28.945348 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0bd2e1-84b6-4d9f-9f57-3ea7f9eceea8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:28.973946 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0bd2e1-84b6-4d9f-9f57-3ea7f9eceea8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:28.975829 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0bd2e1-84b6-4d9f-9f57-3ea7f9eceea8 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:28.998216 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-6b0bd2e1-84b6-4d9f-9f57-3ea7f9eceea8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:23:28.998216 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:28.998216 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:28.998216 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:28.998216 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:28.998216 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:28.998216 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:28.998216 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:28.998216 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:28.998216 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:28.998216 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:28.998216 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:28.998216 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:23:28.998216 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:23:28.998216 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:23:29.000156 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:23:29.000156 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:29.000156 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:29.000156 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:29.000156 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:29.000156 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:29.000156 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:29.000156 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:29.000156 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:29.000156 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:29.000156 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:29.000156 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:29.000156 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:29.000156 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:29.000156 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:29.000156 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:29.000156 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:29.001584 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:23:29.001584 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:23:29.001584 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:23:29.001584 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:23:29.004609 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 375/766] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:23:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 15:23:29.010172 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0bd2e1-84b6-4d9f-9f57-3ea7f9eceea8 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:29.034620 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 392/767] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:23:28 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 15:23:29.035447 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:29.066124 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ae1abb-39eb-4ee0-bb9c-41f37f402bc4 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:29.066423 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ae1abb-39eb-4ee0-bb9c-41f37f402bc4 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:29.066617 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ae1abb-39eb-4ee0-bb9c-41f37f402bc4 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:29.068240 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 393/768] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:23:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:23:29.074087 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c2ae6e-e547-47c4-8f11-940cb1735f97 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:29.074087 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c2ae6e-e547-47c4-8f11-940cb1735f97 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:29.074087 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c2ae6e-e547-47c4-8f11-940cb1735f97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:29.100106 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-49c2ae6e-e547-47c4-8f11-940cb1735f97 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:23:29.101689 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-49c2ae6e-e547-47c4-8f11-940cb1735f97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:23:29.136469 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0bd2e1-84b6-4d9f-9f57-3ea7f9eceea8 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:29.141691 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:29.176943 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa09863-7650-4293-8ec1-2e1f4ed2d172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:29.178246 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa09863-7650-4293-8ec1-2e1f4ed2d172 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:29.179090 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa09863-7650-4293-8ec1-2e1f4ed2d172 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:29.179559 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa09863-7650-4293-8ec1-2e1f4ed2d172 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:29.180574 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa09863-7650-4293-8ec1-2e1f4ed2d172 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:29.181073 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa09863-7650-4293-8ec1-2e1f4ed2d172 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:29.183567 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa09863-7650-4293-8ec1-2e1f4ed2d172 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:29.208316 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 394/769] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:23:29 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 15:23:29.209134 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c2ae6e-e547-47c4-8f11-940cb1735f97 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:29.213318 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 376/770] 10.222.0.99 () {56 vars in 1109 bytes} [Thu Jul 14 15:23:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:29.218494 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa09863-7650-4293-8ec1-2e1f4ed2d172 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:29.311526 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:29.335440 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c2ae6e-e547-47c4-8f11-940cb1735f97 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:29.339138 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa09863-7650-4293-8ec1-2e1f4ed2d172 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:29.374418 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abad8476-89a6-4a75-9c00-aa51f072ff69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:29.375545 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abad8476-89a6-4a75-9c00-aa51f072ff69 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:29.376047 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abad8476-89a6-4a75-9c00-aa51f072ff69 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:29.376565 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abad8476-89a6-4a75-9c00-aa51f072ff69 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:29.384273 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e309b8d6-48ac-4b7b-a9eb-7738c78ac43f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:29.385819 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e309b8d6-48ac-4b7b-a9eb-7738c78ac43f None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:29.386204 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e309b8d6-48ac-4b7b-a9eb-7738c78ac43f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:29.386563 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e309b8d6-48ac-4b7b-a9eb-7738c78ac43f None admin] PATH_INFO: `/v3/users/nova` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:29.389315 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abad8476-89a6-4a75-9c00-aa51f072ff69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:23:29.395828 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e309b8d6-48ac-4b7b-a9eb-7738c78ac43f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:29.397295 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e309b8d6-48ac-4b7b-a9eb-7738c78ac43f None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:29.398535 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-abad8476-89a6-4a75-9c00-aa51f072ff69 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 15:23:29.398535 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:29.398535 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:29.398535 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:29.398535 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:29.398535 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:29.398535 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:29.398535 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:29.398535 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:29.398535 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:29.398535 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:29.398535 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:29.398535 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:23:29.398535 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:23:29.398535 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:23:29.398535 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:23:29.399548 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:23:29.399548 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:23:29.399548 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:23:29.399548 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:23:29.399548 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:29.399548 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:29.399548 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:29.399548 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:29.399548 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:29.399548 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:29.399548 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:29.399548 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:29.399548 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:29.399548 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:29.399548 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:29.399548 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:29.400409 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:29.400409 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:29.400409 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:29.400409 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:29.400409 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:23:29.400409 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:23:29.400409 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:23:29.400409 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:23:29.400409 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:23:29.400409 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:23:29.400409 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 15:23:29.400409 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:23:29.403176 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 395/771] 10.222.0.99 () {56 vars in 1102 bytes} [Thu Jul 14 15:23:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:29.404298 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abad8476-89a6-4a75-9c00-aa51f072ff69 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:29.424763 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-e309b8d6-48ac-4b7b-a9eb-7738c78ac43f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 14 15:23:29.424763 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:29.424763 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:29.424763 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:29.424763 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:29.424763 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:29.424763 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:29.424763 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:29.424763 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:29.424763 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:29.424763 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:29.424763 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:29.424763 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 15:23:29.424763 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 15:23:29.424763 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 15:23:29.424763 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 15:23:29.425748 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:29.425748 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:29.425748 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 15:23:29.425748 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:23:29.425748 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 15:23:29.425748 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:23:29.425748 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:29.425748 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:29.425748 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:29.425748 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:29.425748 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:29.425748 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:29.425748 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:29.425748 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:29.425748 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:29.425748 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:29.425748 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:29.425748 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:29.426665 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:29.426665 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:29.426665 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 15:23:29.426665 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 15:23:29.426665 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 15:23:29.426665 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 15:23:29.426665 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 15:23:29.426665 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 15:23:29.426665 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 14 15:23:29.426665 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:23:29.427767 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 377/772] 10.222.0.99 () {56 vars in 1093 bytes} [Thu Jul 14 15:23:29 2022] GET /identity/v3/users/nova => generated 82 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:29.431223 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e309b8d6-48ac-4b7b-a9eb-7738c78ac43f None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:29.433158 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:29.542856 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e309b8d6-48ac-4b7b-a9eb-7738c78ac43f None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:29.554322 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abad8476-89a6-4a75-9c00-aa51f072ff69 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:29.581960 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ec9910-05f6-45a4-9c7c-2bbffaa7ea71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:29.582648 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ec9910-05f6-45a4-9c7c-2bbffaa7ea71 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:29.582917 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ec9910-05f6-45a4-9c7c-2bbffaa7ea71 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:29.583201 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ec9910-05f6-45a4-9c7c-2bbffaa7ea71 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:29.584047 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ec9910-05f6-45a4-9c7c-2bbffaa7ea71 None admin] RBAC: Adding query filter params (name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:29.584360 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ec9910-05f6-45a4-9c7c-2bbffaa7ea71 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:29.585983 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ec9910-05f6-45a4-9c7c-2bbffaa7ea71 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:29.588375 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:29.601694 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 378/773] 10.222.0.99 () {56 vars in 1103 bytes} [Thu Jul 14 15:23:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:29.602650 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ec9910-05f6-45a4-9c7c-2bbffaa7ea71 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:29.613329 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e00a711-c2f1-4d20-a8a2-81e4ed7ff118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:29.614844 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e00a711-c2f1-4d20-a8a2-81e4ed7ff118 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:29.615469 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e00a711-c2f1-4d20-a8a2-81e4ed7ff118 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:29.616052 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e00a711-c2f1-4d20-a8a2-81e4ed7ff118 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:29.617164 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e00a711-c2f1-4d20-a8a2-81e4ed7ff118 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:29.619840 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e00a711-c2f1-4d20-a8a2-81e4ed7ff118 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:29.634072 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-9e00a711-c2f1-4d20-a8a2-81e4ed7ff118 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 14 15:23:29.634072 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:29.634072 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 15:23:29.634072 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 15:23:29.634072 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 15:23:29.634072 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 15:23:29.634072 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 15:23:29.634072 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 15:23:29.634072 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 14 15:23:29.634072 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:23:29.634072 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 15:23:29.634072 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:23:29.634072 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:29.634072 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:29.634072 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:29.634072 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:29.635515 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:29.635515 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:29.635515 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:29.635515 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:29.635515 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:29.635515 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:29.635515 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:29.635515 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 15:23:29.635515 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 15:23:29.635515 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 15:23:29.635515 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 15:23:29.635515 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:29.635515 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:29.635515 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:29.635515 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:29.635515 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:29.636544 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:29.636544 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:29.636544 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:29.636544 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:29.636544 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:29.636544 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:29.636544 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:29.636544 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:29.636544 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:29.636544 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:29.636544 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:29.636544 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 15:23:29.636544 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 15:23:29.636544 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 14 15:23:29.636544 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:23:29.638951 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 396/774] 10.222.0.99 () {56 vars in 1129 bytes} [Thu Jul 14 15:23:29 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:29.640008 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e00a711-c2f1-4d20-a8a2-81e4ed7ff118 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:29.723421 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ec9910-05f6-45a4-9c7c-2bbffaa7ea71 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:29.738105 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:29.756871 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e00a711-c2f1-4d20-a8a2-81e4ed7ff118 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:29.761558 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415a8c1d-a3bc-46b0-82d9-4c03d0a1bf2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:29.762241 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415a8c1d-a3bc-46b0-82d9-4c03d0a1bf2d None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:29.762532 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415a8c1d-a3bc-46b0-82d9-4c03d0a1bf2d None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:29.762815 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415a8c1d-a3bc-46b0-82d9-4c03d0a1bf2d None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:29.763696 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415a8c1d-a3bc-46b0-82d9-4c03d0a1bf2d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:29.764014 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415a8c1d-a3bc-46b0-82d9-4c03d0a1bf2d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:29.765500 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415a8c1d-a3bc-46b0-82d9-4c03d0a1bf2d None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:29.785016 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 379/775] 10.222.0.99 () {56 vars in 1133 bytes} [Thu Jul 14 15:23:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:29.785969 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415a8c1d-a3bc-46b0-82d9-4c03d0a1bf2d None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:29.797342 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fcbf1d-d7f7-4e76-b604-941c3739d9cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:29.798388 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fcbf1d-d7f7-4e76-b604-941c3739d9cf None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:29.798836 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fcbf1d-d7f7-4e76-b604-941c3739d9cf None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:29.799296 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fcbf1d-d7f7-4e76-b604-941c3739d9cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:29.811372 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fcbf1d-d7f7-4e76-b604-941c3739d9cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:29.813471 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fcbf1d-d7f7-4e76-b604-941c3739d9cf None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:29.832553 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-15fcbf1d-d7f7-4e76-b604-941c3739d9cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:23:29.832553 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:29.832553 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:29.832553 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:29.832553 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:29.832553 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:29.832553 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:29.832553 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:29.832553 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:29.832553 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:29.832553 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:29.832553 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:29.832553 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:23:29.832553 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:23:29.832553 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:23:29.833716 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:23:29.833716 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:29.833716 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:29.833716 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:29.833716 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:29.833716 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:29.833716 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:29.833716 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:29.833716 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:29.833716 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:29.833716 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:29.833716 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:29.833716 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:29.833716 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:29.833716 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:29.833716 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:29.833716 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:29.834712 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:23:29.834712 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:23:29.834712 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:23:29.834712 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:23:29.836753 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 397/776] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:23:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:29.837760 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fcbf1d-d7f7-4e76-b604-941c3739d9cf None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:29.855712 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:29.910843 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415a8c1d-a3bc-46b0-82d9-4c03d0a1bf2d None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:29.950272 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1d9fe4-49ab-4b6b-9369-f2fd71444032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:29.950948 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1d9fe4-49ab-4b6b-9369-f2fd71444032 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:29.951239 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1d9fe4-49ab-4b6b-9369-f2fd71444032 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:29.951522 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1d9fe4-49ab-4b6b-9369-f2fd71444032 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:29.952864 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1d9fe4-49ab-4b6b-9369-f2fd71444032 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:29.953176 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1d9fe4-49ab-4b6b-9369-f2fd71444032 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:29.954911 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1d9fe4-49ab-4b6b-9369-f2fd71444032 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:29.965086 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 380/777] 10.222.0.99 () {56 vars in 1124 bytes} [Thu Jul 14 15:23:29 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:29.966032 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1d9fe4-49ab-4b6b-9369-f2fd71444032 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:30.010388 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fcbf1d-d7f7-4e76-b604-941c3739d9cf None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:30.052061 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8691437d-0796-44a6-ab72-5fdebe1f0ab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:30.052868 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8691437d-0796-44a6-ab72-5fdebe1f0ab9 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:30.053151 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8691437d-0796-44a6-ab72-5fdebe1f0ab9 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:30.053426 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8691437d-0796-44a6-ab72-5fdebe1f0ab9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:30.061480 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:30.069809 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8691437d-0796-44a6-ab72-5fdebe1f0ab9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:30.072921 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8691437d-0796-44a6-ab72-5fdebe1f0ab9 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:30.109568 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1d9fe4-49ab-4b6b-9369-f2fd71444032 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:30.133091 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-8691437d-0796-44a6-ab72-5fdebe1f0ab9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:23:30.133091 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:30.133091 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:30.133091 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:30.133091 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:30.133091 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:30.133091 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:30.133091 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:30.133091 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:30.133091 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:30.133091 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:30.133091 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:30.133091 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:23:30.133091 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:23:30.133091 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:23:30.134128 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:23:30.134128 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:30.134128 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:30.134128 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:30.134128 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:30.134128 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:30.134128 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:30.134128 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:30.134128 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:30.134128 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:30.134128 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:30.134128 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:30.134128 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:30.134128 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:30.134128 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:30.134128 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:30.134128 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:30.135338 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:23:30.135338 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:23:30.135338 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:23:30.135338 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:23:30.140460 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 398/778] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:23:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:30.141408 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8691437d-0796-44a6-ab72-5fdebe1f0ab9 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:30.142093 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:30.173589 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52b3e02-5d52-4950-8e4c-d97010fdfac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:30.174269 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52b3e02-5d52-4950-8e4c-d97010fdfac2 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:30.174560 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52b3e02-5d52-4950-8e4c-d97010fdfac2 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:30.174847 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52b3e02-5d52-4950-8e4c-d97010fdfac2 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:30.175575 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52b3e02-5d52-4950-8e4c-d97010fdfac2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:30.175995 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52b3e02-5d52-4950-8e4c-d97010fdfac2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:30.178239 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52b3e02-5d52-4950-8e4c-d97010fdfac2 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:30.206024 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 381/779] 10.222.0.99 () {56 vars in 1109 bytes} [Thu Jul 14 15:23:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:30.206940 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52b3e02-5d52-4950-8e4c-d97010fdfac2 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:30.300749 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8691437d-0796-44a6-ab72-5fdebe1f0ab9 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:30.350966 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e442dfb-0056-4c5c-8e1a-5a4b4d2e20ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:30.351896 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e442dfb-0056-4c5c-8e1a-5a4b4d2e20ed None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:30.352234 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e442dfb-0056-4c5c-8e1a-5a4b4d2e20ed None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:30.352558 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e442dfb-0056-4c5c-8e1a-5a4b4d2e20ed None admin] PATH_INFO: `/v3/services` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:30.353293 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e442dfb-0056-4c5c-8e1a-5a4b4d2e20ed None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:30.353611 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e442dfb-0056-4c5c-8e1a-5a4b4d2e20ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:30.355014 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e442dfb-0056-4c5c-8e1a-5a4b4d2e20ed None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:30.366136 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 399/780] 10.222.0.99 () {56 vars in 1124 bytes} [Thu Jul 14 15:23:30 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:30.367034 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e442dfb-0056-4c5c-8e1a-5a4b4d2e20ed None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:30.429221 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:30.489772 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e442dfb-0056-4c5c-8e1a-5a4b4d2e20ed None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:30.495811 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52b3e02-5d52-4950-8e4c-d97010fdfac2 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:30.527836 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f492bdc5-5632-4023-ba2b-97278217013c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:30.529442 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f492bdc5-5632-4023-ba2b-97278217013c None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:30.529952 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f492bdc5-5632-4023-ba2b-97278217013c None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:30.530858 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f492bdc5-5632-4023-ba2b-97278217013c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:30.540376 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f492bdc5-5632-4023-ba2b-97278217013c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:30.542333 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f492bdc5-5632-4023-ba2b-97278217013c None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:30.544119 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01594908-304d-47f6-a696-aee1ee21a1cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:30.545361 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01594908-304d-47f6-a696-aee1ee21a1cf None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:30.545827 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01594908-304d-47f6-a696-aee1ee21a1cf None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:30.546810 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01594908-304d-47f6-a696-aee1ee21a1cf None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:30.547755 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01594908-304d-47f6-a696-aee1ee21a1cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:30.547979 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01594908-304d-47f6-a696-aee1ee21a1cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:30.549353 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01594908-304d-47f6-a696-aee1ee21a1cf None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:30.559072 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-f492bdc5-5632-4023-ba2b-97278217013c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 15:23:30.559072 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:30.559072 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:30.559072 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:30.559072 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:30.559072 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:30.559072 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:30.559072 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:30.559072 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:30.559072 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:30.559072 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:30.559072 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:30.559072 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 15:23:30.559072 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 15:23:30.559072 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 15:23:30.559072 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 15:23:30.561029 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:30.561029 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:30.561029 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 15:23:30.561029 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:23:30.561029 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 15:23:30.561029 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:23:30.561029 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:30.561029 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:30.561029 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:30.561029 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:30.561029 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:30.561029 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:30.561029 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:30.561029 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:30.561029 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:30.561029 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:30.561029 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:30.561029 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:30.562852 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:30.562852 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:30.562852 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 15:23:30.562852 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 15:23:30.562852 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 15:23:30.562852 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 15:23:30.562852 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 15:23:30.562852 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 15:23:30.562852 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 15:23:30.562852 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:23:30.562852 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 400/781] 10.222.0.99 () {56 vars in 1117 bytes} [Thu Jul 14 15:23:30 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:30.562852 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f492bdc5-5632-4023-ba2b-97278217013c None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:30.574312 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 382/782] 10.222.0.99 () {56 vars in 1109 bytes} [Thu Jul 14 15:23:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:30.574312 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923054f9-4e99-4cf3-9c55-d26ab3d9c533 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:30.574312 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923054f9-4e99-4cf3-9c55-d26ab3d9c533 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:30.574312 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923054f9-4e99-4cf3-9c55-d26ab3d9c533 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:30.574312 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 383/783] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:23:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:23:30.574312 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923054f9-4e99-4cf3-9c55-d26ab3d9c533 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:30.623055 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:30.674184 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f492bdc5-5632-4023-ba2b-97278217013c None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:30.689644 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:30.693220 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923054f9-4e99-4cf3-9c55-d26ab3d9c533 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:30.712082 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a4363b-afb4-4afc-a346-dac25f61df19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:30.712870 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a4363b-afb4-4afc-a346-dac25f61df19 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:30.713049 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a4363b-afb4-4afc-a346-dac25f61df19 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:30.713301 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a4363b-afb4-4afc-a346-dac25f61df19 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:30.714105 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a4363b-afb4-4afc-a346-dac25f61df19 None admin] RBAC: Adding query filter params (interface=public, service_id=44e08b216b5c4fa5a34c7e9747adc75d, region_id=RegionOne) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:30.714500 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a4363b-afb4-4afc-a346-dac25f61df19 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:30.715964 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a4363b-afb4-4afc-a346-dac25f61df19 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:30.727557 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 401/784] 10.222.0.99 () {56 vars in 1251 bytes} [Thu Jul 14 15:23:30 2022] GET /identity/v3/endpoints?service_id=44e08b216b5c4fa5a34c7e9747adc75d&interface=public®ion_id=RegionOne => generated 194 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:30.728445 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a4363b-afb4-4afc-a346-dac25f61df19 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:30.735570 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4226a97c-9ff1-4ad5-8bff-3fda64a62d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:30.736340 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4226a97c-9ff1-4ad5-8bff-3fda64a62d62 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:30.736875 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4226a97c-9ff1-4ad5-8bff-3fda64a62d62 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:30.737159 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4226a97c-9ff1-4ad5-8bff-3fda64a62d62 None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:30.738011 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4226a97c-9ff1-4ad5-8bff-3fda64a62d62 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:30.738325 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4226a97c-9ff1-4ad5-8bff-3fda64a62d62 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:30.739791 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4226a97c-9ff1-4ad5-8bff-3fda64a62d62 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:30.762197 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 384/785] 10.222.0.99 () {56 vars in 1149 bytes} [Thu Jul 14 15:23:30 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:30.763322 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4530e4b-1f7e-4f0a-9766-7899f1d1e2d2 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:30.763628 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4530e4b-1f7e-4f0a-9766-7899f1d1e2d2 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:30.763913 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4530e4b-1f7e-4f0a-9766-7899f1d1e2d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:30.782809 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-c4530e4b-1f7e-4f0a-9766-7899f1d1e2d2 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:23:30.784080 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c4530e4b-1f7e-4f0a-9766-7899f1d1e2d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:23:30.856701 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a4363b-afb4-4afc-a346-dac25f61df19 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:30.867671 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 385/786] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:23:30 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 15:23:30.867671 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4530e4b-1f7e-4f0a-9766-7899f1d1e2d2 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:30.903871 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:30.905916 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0f64a0-60a1-4caf-a686-a9c8f584b43e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:30.907576 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0f64a0-60a1-4caf-a686-a9c8f584b43e None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:30.907899 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0f64a0-60a1-4caf-a686-a9c8f584b43e None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:30.908510 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0f64a0-60a1-4caf-a686-a9c8f584b43e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:30.916988 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0f64a0-60a1-4caf-a686-a9c8f584b43e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:30.918919 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0f64a0-60a1-4caf-a686-a9c8f584b43e None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:30.935813 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-5e0f64a0-60a1-4caf-a686-a9c8f584b43e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:23:30.935813 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:30.935813 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:30.935813 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:30.935813 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:30.935813 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:30.935813 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:30.935813 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:30.935813 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:30.935813 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:30.935813 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:30.935813 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:30.935813 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 15:23:30.935813 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 15:23:30.935813 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 15:23:30.937464 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 15:23:30.937464 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:30.937464 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:30.937464 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:30.937464 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:30.937464 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:30.937464 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:30.937464 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:30.937464 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:30.937464 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:30.937464 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:30.937464 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:30.937464 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:30.937464 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:30.937464 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:30.937464 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:30.937464 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:30.938344 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 15:23:30.938344 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 15:23:30.938344 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 15:23:30.938344 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 15:23:30.938344 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 15:23:30.938344 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 15:23:30.938344 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:23:30.938344 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:23:30.938344 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 402/787] 10.222.0.99 () {56 vars in 1111 bytes} [Thu Jul 14 15:23:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:30.939496 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0f64a0-60a1-4caf-a686-a9c8f584b43e None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:30.998628 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:31.001989 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4530e4b-1f7e-4f0a-9766-7899f1d1e2d2 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:31.045180 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9640cb2d-90cf-4d38-9687-5f54f708ae2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:31.046725 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9640cb2d-90cf-4d38-9687-5f54f708ae2f None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:31.047159 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9640cb2d-90cf-4d38-9687-5f54f708ae2f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:31.047635 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9640cb2d-90cf-4d38-9687-5f54f708ae2f None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:31.048466 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9640cb2d-90cf-4d38-9687-5f54f708ae2f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:31.049989 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9640cb2d-90cf-4d38-9687-5f54f708ae2f None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:31.055666 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 386/788] 10.222.0.99 () {56 vars in 1087 bytes} [Thu Jul 14 15:23:30 2022] GET /identity/v3/services => generated 772 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:31.056615 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9640cb2d-90cf-4d38-9687-5f54f708ae2f None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:31.089192 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0f64a0-60a1-4caf-a686-a9c8f584b43e None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:31.151297 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3abec72-6c48-4577-aa79-e47ad2a4b73f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:31.152023 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3abec72-6c48-4577-aa79-e47ad2a4b73f None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:31.152214 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3abec72-6c48-4577-aa79-e47ad2a4b73f None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:31.152764 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3abec72-6c48-4577-aa79-e47ad2a4b73f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:31.174248 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3abec72-6c48-4577-aa79-e47ad2a4b73f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:31.175699 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3abec72-6c48-4577-aa79-e47ad2a4b73f None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:31.196387 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-c3abec72-6c48-4577-aa79-e47ad2a4b73f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:23:31.196387 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:31.196387 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:31.196387 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:31.196387 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:31.196387 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:31.196387 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:31.196387 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:31.196387 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:31.196387 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:31.196387 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:31.196387 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:31.196387 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:23:31.196387 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:23:31.196387 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:23:31.197365 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:23:31.197365 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:31.197365 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:31.197365 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:31.197365 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:31.197365 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:31.197365 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:31.197365 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:31.197365 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:31.197365 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:31.197365 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:31.197365 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:31.197365 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:31.197365 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:31.197365 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:31.197365 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:31.197365 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:31.198361 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:23:31.198361 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:23:31.198361 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:23:31.198361 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:23:31.198361 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 403/789] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:23:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:31.199441 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3abec72-6c48-4577-aa79-e47ad2a4b73f None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:31.204109 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:31.217172 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9640cb2d-90cf-4d38-9687-5f54f708ae2f None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:31.261965 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57143ae3-84f5-4b8c-8a6d-ae45a63797c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:31.262795 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57143ae3-84f5-4b8c-8a6d-ae45a63797c5 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:31.263083 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57143ae3-84f5-4b8c-8a6d-ae45a63797c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:31.263357 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57143ae3-84f5-4b8c-8a6d-ae45a63797c5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:31.275728 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57143ae3-84f5-4b8c-8a6d-ae45a63797c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:23:31.284735 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-57143ae3-84f5-4b8c-8a6d-ae45a63797c5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:23:31.284735 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:31.284735 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:31.284735 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:31.284735 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:31.284735 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:31.284735 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:31.284735 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:31.284735 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:31.284735 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:31.284735 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:31.284735 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:31.284735 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:23:31.284735 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:23:31.284735 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:23:31.284735 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:23:31.285744 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:23:31.285744 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:23:31.285744 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:23:31.285744 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:23:31.285744 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:31.285744 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:31.285744 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:31.285744 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:31.285744 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:31.285744 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:31.285744 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:31.285744 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:31.285744 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:31.285744 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:31.285744 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:31.285744 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:31.286577 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:31.286577 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:31.286577 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:31.286577 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:31.286577 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:23:31.286577 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:23:31.286577 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:23:31.286577 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:23:31.286577 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:23:31.286577 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:23:31.286577 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:23:31.286577 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:23:31.286577 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 387/790] 10.222.0.99 () {56 vars in 1096 bytes} [Thu Jul 14 15:23:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:31.287918 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef19bb7-0c08-42c2-b6c4-dabf4d542fff None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:31.287918 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef19bb7-0c08-42c2-b6c4-dabf4d542fff None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:31.288081 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef19bb7-0c08-42c2-b6c4-dabf4d542fff None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:31.291716 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 388/791] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:23:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:23:31.291716 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef19bb7-0c08-42c2-b6c4-dabf4d542fff None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:31.346470 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:31.356784 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3abec72-6c48-4577-aa79-e47ad2a4b73f None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:31.396284 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b8a8ec-3af0-4a22-a0a6-d53f82bd68af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:31.396968 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b8a8ec-3af0-4a22-a0a6-d53f82bd68af None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:31.397254 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b8a8ec-3af0-4a22-a0a6-d53f82bd68af None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:31.397535 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b8a8ec-3af0-4a22-a0a6-d53f82bd68af None admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:31.398340 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b8a8ec-3af0-4a22-a0a6-d53f82bd68af None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:31.398655 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b8a8ec-3af0-4a22-a0a6-d53f82bd68af None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:31.400186 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b8a8ec-3af0-4a22-a0a6-d53f82bd68af None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:31.413899 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 404/792] 10.222.0.99 () {56 vars in 1148 bytes} [Thu Jul 14 15:23:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:31.414203 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef19bb7-0c08-42c2-b6c4-dabf4d542fff None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:31.419271 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b8a8ec-3af0-4a22-a0a6-d53f82bd68af None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:31.455895 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8095eb65-6847-4ca5-b2b5-6e2db5e06ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:31.456466 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8095eb65-6847-4ca5-b2b5-6e2db5e06ebb None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:31.456745 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8095eb65-6847-4ca5-b2b5-6e2db5e06ebb None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:31.457021 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8095eb65-6847-4ca5-b2b5-6e2db5e06ebb None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:31.457747 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8095eb65-6847-4ca5-b2b5-6e2db5e06ebb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:31.458054 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8095eb65-6847-4ca5-b2b5-6e2db5e06ebb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:31.459656 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8095eb65-6847-4ca5-b2b5-6e2db5e06ebb None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:31.473476 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 389/793] 10.222.0.99 () {56 vars in 1109 bytes} [Thu Jul 14 15:23:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:31.480033 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acc0001-1db0-43d8-a3c4-bde244f58c5d None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:31.480033 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acc0001-1db0-43d8-a3c4-bde244f58c5d None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:31.480033 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acc0001-1db0-43d8-a3c4-bde244f58c5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:31.496731 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-0acc0001-1db0-43d8-a3c4-bde244f58c5d None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:23:31.497948 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0acc0001-1db0-43d8-a3c4-bde244f58c5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:23:31.507140 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:31.548847 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b8a8ec-3af0-4a22-a0a6-d53f82bd68af None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:31.587668 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 390/794] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:23:31 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 15:23:31.588769 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acc0001-1db0-43d8-a3c4-bde244f58c5d None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:31.590733 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe00a21d-3e41-4b79-a750-d25fba475129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:31.591718 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe00a21d-3e41-4b79-a750-d25fba475129 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:31.592240 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe00a21d-3e41-4b79-a750-d25fba475129 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:31.593373 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe00a21d-3e41-4b79-a750-d25fba475129 None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:31.594485 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe00a21d-3e41-4b79-a750-d25fba475129 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:31.594999 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe00a21d-3e41-4b79-a750-d25fba475129 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:31.596690 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe00a21d-3e41-4b79-a750-d25fba475129 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:31.611512 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 405/795] 10.222.0.99 () {56 vars in 1099 bytes} [Thu Jul 14 15:23:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:31.612698 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe00a21d-3e41-4b79-a750-d25fba475129 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:31.630422 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:31.701455 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acc0001-1db0-43d8-a3c4-bde244f58c5d None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:31.741795 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c751d35-6ace-42e6-8390-0a78c049e056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:31.742703 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c751d35-6ace-42e6-8390-0a78c049e056 None admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:31.742997 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c751d35-6ace-42e6-8390-0a78c049e056 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:31.743282 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c751d35-6ace-42e6-8390-0a78c049e056 None admin] PATH_INFO: `/v3/projects/9048d1a6f00443ec92f11471babe5a56/users/3a9b1c2622934c5185f3b5f1ea623bc0/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:31.745319 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c751d35-6ace-42e6-8390-0a78c049e056 None admin] RBAC: Authorizing `identity:create_grant(project_id=9048d1a6f00443ec92f11471babe5a56, user_id=3a9b1c2622934c5185f3b5f1ea623bc0, role_id=d0f6ba1314f14d9f9c816bdb5d0c3aac)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:31.746201 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe00a21d-3e41-4b79-a750-d25fba475129 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:31.746761 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c751d35-6ace-42e6-8390-0a78c049e056 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:31.763302 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 391/796] 10.222.0.99 () {58 vars in 1439 bytes} [Thu Jul 14 15:23:31 2022] PUT /identity/v3/projects/9048d1a6f00443ec92f11471babe5a56/users/3a9b1c2622934c5185f3b5f1ea623bc0/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:23:31.765913 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c751d35-6ace-42e6-8390-0a78c049e056 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:31.786959 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:31.799470 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d92be86-02b6-44cc-8926-f09aa28e8329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:31.800353 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d92be86-02b6-44cc-8926-f09aa28e8329 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:31.800644 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d92be86-02b6-44cc-8926-f09aa28e8329 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:31.800921 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d92be86-02b6-44cc-8926-f09aa28e8329 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:31.812361 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d92be86-02b6-44cc-8926-f09aa28e8329 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:31.814608 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d92be86-02b6-44cc-8926-f09aa28e8329 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:31.834910 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-4d92be86-02b6-44cc-8926-f09aa28e8329 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:23:31.834910 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:31.834910 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:31.834910 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:31.834910 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:31.834910 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:31.834910 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:31.834910 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:31.834910 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:31.834910 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:31.834910 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:31.834910 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:31.834910 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 15:23:31.834910 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 15:23:31.834910 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 15:23:31.835888 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 15:23:31.835888 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:31.835888 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:31.835888 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:31.835888 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:31.835888 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:31.835888 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:31.835888 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:31.835888 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:31.835888 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:31.835888 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:31.835888 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:31.835888 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:31.835888 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:31.835888 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:31.835888 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:31.835888 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:31.836821 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 15:23:31.836821 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 15:23:31.836821 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 15:23:31.836821 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 15:23:31.836821 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 15:23:31.836821 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 15:23:31.836821 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:23:31.836821 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:23:31.837253 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 406/797] 10.222.0.99 () {56 vars in 1111 bytes} [Thu Jul 14 15:23:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:31.837921 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d92be86-02b6-44cc-8926-f09aa28e8329 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:31.914881 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c751d35-6ace-42e6-8390-0a78c049e056 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:31.939133 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:31.961595 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fb6919-80f2-427d-9ccb-775500a5fb3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:31.962404 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fb6919-80f2-427d-9ccb-775500a5fb3a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:31.962689 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fb6919-80f2-427d-9ccb-775500a5fb3a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:31.962966 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fb6919-80f2-427d-9ccb-775500a5fb3a None admin] PATH_INFO: `/v3/services/network` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:31.963652 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fb6919-80f2-427d-9ccb-775500a5fb3a None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:31.965179 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fb6919-80f2-427d-9ccb-775500a5fb3a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:31.977639 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d92be86-02b6-44cc-8926-f09aa28e8329 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:31.988027 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-80fb6919-80f2-427d-9ccb-775500a5fb3a None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 14 15:23:31.988027 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:31.988027 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 15:23:31.988027 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 15:23:31.988027 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 15:23:31.988027 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 15:23:31.988027 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 15:23:31.988027 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 15:23:31.988027 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 14 15:23:31.988027 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:23:31.988027 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 15:23:31.988027 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:23:31.988027 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:31.988027 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:31.988027 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:31.988027 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:31.988027 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:31.989302 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:31.989302 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:31.989302 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:31.989302 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:31.989302 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:31.989302 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:31.989302 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 15:23:31.989302 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 15:23:31.989302 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 15:23:31.989302 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 15:23:31.989302 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:31.989302 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:31.989302 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:31.989302 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:31.989302 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:31.989302 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:31.990367 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:31.990367 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:31.990367 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:31.990367 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:31.990367 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:31.990367 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:31.990367 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:31.990367 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:31.990367 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:31.990367 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:31.990367 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 15:23:31.990367 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 15:23:31.990367 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 14 15:23:31.990367 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:23:31.990367 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 392/798] 10.222.0.99 () {56 vars in 1111 bytes} [Thu Jul 14 15:23:31 2022] GET /identity/v3/services/network => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:31.991202 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fb6919-80f2-427d-9ccb-775500a5fb3a None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:32.029905 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eca96f4-0e44-4c63-955f-f469bc66fdae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:32.032130 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eca96f4-0e44-4c63-955f-f469bc66fdae None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:32.032656 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eca96f4-0e44-4c63-955f-f469bc66fdae None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:32.033182 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eca96f4-0e44-4c63-955f-f469bc66fdae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:32.045493 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eca96f4-0e44-4c63-955f-f469bc66fdae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:32.048604 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eca96f4-0e44-4c63-955f-f469bc66fdae None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:32.068605 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:32.071614 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-5eca96f4-0e44-4c63-955f-f469bc66fdae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:23:32.071614 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:32.071614 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:32.071614 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:32.071614 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:32.071614 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:32.071614 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:32.071614 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:32.071614 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:32.071614 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:32.071614 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:32.071614 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:32.071614 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:23:32.071614 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:23:32.071614 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:23:32.072567 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:23:32.072567 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:32.072567 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:32.072567 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:32.072567 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:32.072567 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:32.072567 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:32.072567 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:32.072567 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:32.072567 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:32.072567 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:32.072567 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:32.072567 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:32.072567 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:32.072567 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:32.072567 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:32.072567 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:32.073447 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:23:32.073447 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:23:32.073447 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:23:32.073447 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:23:32.076324 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 407/799] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:23:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:32.078325 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eca96f4-0e44-4c63-955f-f469bc66fdae None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:32.109488 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fb6919-80f2-427d-9ccb-775500a5fb3a None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:32.146513 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6b9261-6e64-4dad-97da-2b8a79107f3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:32.147192 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6b9261-6e64-4dad-97da-2b8a79107f3e None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:32.147485 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6b9261-6e64-4dad-97da-2b8a79107f3e None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:32.147801 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6b9261-6e64-4dad-97da-2b8a79107f3e None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:32.148699 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6b9261-6e64-4dad-97da-2b8a79107f3e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:32.149021 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6b9261-6e64-4dad-97da-2b8a79107f3e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:32.150467 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6b9261-6e64-4dad-97da-2b8a79107f3e None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:32.165190 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 393/800] 10.222.0.99 () {56 vars in 1148 bytes} [Thu Jul 14 15:23:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:32.166072 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6b9261-6e64-4dad-97da-2b8a79107f3e None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:32.200386 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eca96f4-0e44-4c63-955f-f469bc66fdae None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:32.242848 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:32.261114 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011b4a03-95fa-4899-a78e-67c12b485139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:32.262757 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011b4a03-95fa-4899-a78e-67c12b485139 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:32.263755 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011b4a03-95fa-4899-a78e-67c12b485139 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:32.264384 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011b4a03-95fa-4899-a78e-67c12b485139 None admin] PATH_INFO: `/v3/services` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:32.266342 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011b4a03-95fa-4899-a78e-67c12b485139 None admin] RBAC: Adding query filter params (name=network) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:32.267173 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011b4a03-95fa-4899-a78e-67c12b485139 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:32.269662 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011b4a03-95fa-4899-a78e-67c12b485139 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:32.277020 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 408/801] 10.222.0.99 () {56 vars in 1112 bytes} [Thu Jul 14 15:23:32 2022] GET /identity/v3/services?name=network => generated 124 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:32.278396 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011b4a03-95fa-4899-a78e-67c12b485139 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:32.308084 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6b9261-6e64-4dad-97da-2b8a79107f3e None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:32.338125 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:32.348194 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd42451c-fa7a-49d6-b2a9-1fa526a9af05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:32.349075 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd42451c-fa7a-49d6-b2a9-1fa526a9af05 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:32.349418 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd42451c-fa7a-49d6-b2a9-1fa526a9af05 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:32.349703 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd42451c-fa7a-49d6-b2a9-1fa526a9af05 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:32.350618 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd42451c-fa7a-49d6-b2a9-1fa526a9af05 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:32.350943 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd42451c-fa7a-49d6-b2a9-1fa526a9af05 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:32.352350 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd42451c-fa7a-49d6-b2a9-1fa526a9af05 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:32.366385 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 394/802] 10.222.0.99 () {56 vars in 1109 bytes} [Thu Jul 14 15:23:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:32.367377 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd42451c-fa7a-49d6-b2a9-1fa526a9af05 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:32.401577 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011b4a03-95fa-4899-a78e-67c12b485139 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:32.440484 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3887395f-f185-4ccc-ba56-f4ad6a4cf01a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:32.441735 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3887395f-f185-4ccc-ba56-f4ad6a4cf01a None admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:32.442227 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3887395f-f185-4ccc-ba56-f4ad6a4cf01a None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:32.442732 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3887395f-f185-4ccc-ba56-f4ad6a4cf01a None admin] PATH_INFO: `/v3/projects/9048d1a6f00443ec92f11471babe5a56/users/adaeadf613954d7393b3fd3bae68c362/roles/74e3b597938f49d095cb424c54b5a811` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:32.459245 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3887395f-f185-4ccc-ba56-f4ad6a4cf01a None admin] RBAC: Authorizing `identity:create_grant(project_id=9048d1a6f00443ec92f11471babe5a56, user_id=adaeadf613954d7393b3fd3bae68c362, role_id=74e3b597938f49d095cb424c54b5a811)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:32.461107 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3887395f-f185-4ccc-ba56-f4ad6a4cf01a None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:32.481967 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 409/803] 10.222.0.99 () {58 vars in 1439 bytes} [Thu Jul 14 15:23:32 2022] PUT /identity/v3/projects/9048d1a6f00443ec92f11471babe5a56/users/adaeadf613954d7393b3fd3bae68c362/roles/74e3b597938f49d095cb424c54b5a811 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:23:32.483351 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3887395f-f185-4ccc-ba56-f4ad6a4cf01a None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:32.489334 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd42451c-fa7a-49d6-b2a9-1fa526a9af05 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:32.536470 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664243d4-ad71-49ab-91c7-fb14058dfab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:32.537079 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664243d4-ad71-49ab-91c7-fb14058dfab3 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:32.537370 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664243d4-ad71-49ab-91c7-fb14058dfab3 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:32.537653 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664243d4-ad71-49ab-91c7-fb14058dfab3 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:32.538388 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664243d4-ad71-49ab-91c7-fb14058dfab3 None admin] RBAC: Adding query filter params (type=network) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:32.538701 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664243d4-ad71-49ab-91c7-fb14058dfab3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:32.540255 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664243d4-ad71-49ab-91c7-fb14058dfab3 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:32.541769 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:32.551662 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 395/804] 10.222.0.99 () {56 vars in 1112 bytes} [Thu Jul 14 15:23:32 2022] GET /identity/v3/services?type=network => generated 350 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:32.556697 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664243d4-ad71-49ab-91c7-fb14058dfab3 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:32.677406 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:32.707292 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3887395f-f185-4ccc-ba56-f4ad6a4cf01a None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:32.745572 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664243d4-ad71-49ab-91c7-fb14058dfab3 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:32.754922 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7384566-b3cc-4b3d-a036-ef997d8f1906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:32.755784 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7384566-b3cc-4b3d-a036-ef997d8f1906 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:32.756091 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7384566-b3cc-4b3d-a036-ef997d8f1906 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:32.756395 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7384566-b3cc-4b3d-a036-ef997d8f1906 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:32.766762 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7384566-b3cc-4b3d-a036-ef997d8f1906 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:32.768279 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7384566-b3cc-4b3d-a036-ef997d8f1906 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:32.786711 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-a7384566-b3cc-4b3d-a036-ef997d8f1906 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 14 15:23:32.786711 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:32.786711 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:32.786711 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:32.786711 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:32.786711 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:32.786711 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:32.786711 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:32.786711 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:32.786711 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:32.786711 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:32.786711 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:32.786711 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 15:23:32.786711 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 15:23:32.786711 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 15:23:32.786711 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 15:23:32.787729 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:32.787729 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:32.787729 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 15:23:32.787729 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:23:32.787729 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 15:23:32.787729 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:23:32.787729 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:32.787729 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:32.787729 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:32.787729 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:32.787729 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:32.787729 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:32.787729 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:32.787729 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:32.787729 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:32.787729 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:32.787729 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:32.787729 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:32.788698 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:32.788698 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:32.788698 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 15:23:32.788698 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 15:23:32.788698 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 15:23:32.788698 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 15:23:32.788698 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 15:23:32.788698 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 15:23:32.788698 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 14 15:23:32.788698 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:23:32.789828 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 410/805] 10.222.0.99 () {56 vars in 1096 bytes} [Thu Jul 14 15:23:32 2022] GET /identity/v3/users/swift => generated 83 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:32.796952 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7384566-b3cc-4b3d-a036-ef997d8f1906 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:32.797434 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f8e1ae-eb1e-4a10-829c-e4536d9d9a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:32.798282 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f8e1ae-eb1e-4a10-829c-e4536d9d9a45 None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:32.798730 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f8e1ae-eb1e-4a10-829c-e4536d9d9a45 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:32.799155 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f8e1ae-eb1e-4a10-829c-e4536d9d9a45 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:32.800201 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f8e1ae-eb1e-4a10-829c-e4536d9d9a45 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:32.802082 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f8e1ae-eb1e-4a10-829c-e4536d9d9a45 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:32.842878 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 396/806] 10.222.0.99 () {60 vars in 1144 bytes} [Thu Jul 14 15:23:32 2022] POST /identity/v3/endpoints => generated 327 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 15:23:32.925506 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:32.930694 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7384566-b3cc-4b3d-a036-ef997d8f1906 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:32.979318 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb96b07-a931-4701-bf56-014afb420209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:32.980916 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb96b07-a931-4701-bf56-014afb420209 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:32.981437 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb96b07-a931-4701-bf56-014afb420209 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:32.982012 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb96b07-a931-4701-bf56-014afb420209 None admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:32.983167 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb96b07-a931-4701-bf56-014afb420209 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:32.983810 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb96b07-a931-4701-bf56-014afb420209 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:32.985605 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb96b07-a931-4701-bf56-014afb420209 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:33.017455 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 411/807] 10.222.0.99 () {56 vars in 1135 bytes} [Thu Jul 14 15:23:32 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:33.024329 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f8e1ae-eb1e-4a10-829c-e4536d9d9a45 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:33.177910 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f8e1ae-eb1e-4a10-829c-e4536d9d9a45 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:33.185521 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:33.220751 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec46473-9e37-4b7e-9f79-7e611ed52818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:33.221626 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec46473-9e37-4b7e-9f79-7e611ed52818 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:33.221919 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec46473-9e37-4b7e-9f79-7e611ed52818 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:33.222201 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec46473-9e37-4b7e-9f79-7e611ed52818 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:33.234591 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec46473-9e37-4b7e-9f79-7e611ed52818 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:33.236170 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec46473-9e37-4b7e-9f79-7e611ed52818 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:33.255319 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-cec46473-9e37-4b7e-9f79-7e611ed52818 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:23:33.255319 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:33.255319 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:33.255319 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:33.255319 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:33.255319 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:33.255319 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:33.255319 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:33.255319 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:33.255319 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:33.255319 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:33.255319 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:33.255319 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:23:33.255319 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:23:33.255319 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:23:33.256288 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:23:33.256288 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:33.256288 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:33.256288 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:33.256288 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:33.256288 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:33.256288 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:33.256288 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:33.256288 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:33.256288 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:33.256288 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:33.256288 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:33.256288 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:33.256288 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:33.256288 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:33.256288 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:33.256288 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:33.257156 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:23:33.257156 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:23:33.257156 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:23:33.257156 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:23:33.259468 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 397/808] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:23:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:33.266324 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb96b07-a931-4701-bf56-014afb420209 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:33.382930 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb96b07-a931-4701-bf56-014afb420209 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:33.427989 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242867ed-d1b8-4dac-ad9a-53850caefb43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:33.429045 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242867ed-d1b8-4dac-ad9a-53850caefb43 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:33.429538 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242867ed-d1b8-4dac-ad9a-53850caefb43 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:33.430033 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242867ed-d1b8-4dac-ad9a-53850caefb43 None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:33.430971 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242867ed-d1b8-4dac-ad9a-53850caefb43 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:33.431505 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242867ed-d1b8-4dac-ad9a-53850caefb43 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:33.433193 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242867ed-d1b8-4dac-ad9a-53850caefb43 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:33.449540 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 412/809] 10.222.0.99 () {56 vars in 1109 bytes} [Thu Jul 14 15:23:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:33.455823 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec46473-9e37-4b7e-9f79-7e611ed52818 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:33.457253 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7806513-ceb0-4ef2-b812-e6240a449ed8 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:33.457564 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7806513-ceb0-4ef2-b812-e6240a449ed8 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:33.457836 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7806513-ceb0-4ef2-b812-e6240a449ed8 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:33.459906 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 413/810] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:23:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:23:33.467416 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33717278-c56b-48e3-b83c-8f0a20b2c485 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:33.468308 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33717278-c56b-48e3-b83c-8f0a20b2c485 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:33.468765 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33717278-c56b-48e3-b83c-8f0a20b2c485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:33.493824 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-33717278-c56b-48e3-b83c-8f0a20b2c485 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:23:33.495159 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-33717278-c56b-48e3-b83c-8f0a20b2c485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:23:33.503942 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:33.574809 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec46473-9e37-4b7e-9f79-7e611ed52818 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:33.580098 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 414/811] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:23:33 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 15:23:33.598723 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33717278-c56b-48e3-b83c-8f0a20b2c485 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:33.613804 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8189a7d7-a27b-47e5-9850-ec413afb6c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:33.615065 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8189a7d7-a27b-47e5-9850-ec413afb6c7a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:33.615555 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8189a7d7-a27b-47e5-9850-ec413afb6c7a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:33.616133 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8189a7d7-a27b-47e5-9850-ec413afb6c7a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:33.627233 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8189a7d7-a27b-47e5-9850-ec413afb6c7a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:33.628752 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8189a7d7-a27b-47e5-9850-ec413afb6c7a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:33.646505 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-8189a7d7-a27b-47e5-9850-ec413afb6c7a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:23:33.646505 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:33.646505 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:33.646505 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:33.646505 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:33.646505 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:33.646505 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:33.646505 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:33.646505 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:33.646505 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:33.646505 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:33.646505 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:33.646505 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 15:23:33.646505 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 15:23:33.646505 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 15:23:33.647445 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 15:23:33.647445 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:33.647445 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:33.647445 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:33.647445 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:33.647445 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:33.647445 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:33.647445 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:33.647445 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:33.647445 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:33.647445 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:33.647445 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:33.647445 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:33.647445 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:33.647445 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:33.647445 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:33.647445 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:33.648359 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 15:23:33.648359 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 15:23:33.648359 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 15:23:33.648359 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 15:23:33.648359 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 15:23:33.648359 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 15:23:33.648359 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:23:33.648359 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:23:33.648359 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 398/812] 10.222.0.99 () {56 vars in 1111 bytes} [Thu Jul 14 15:23:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:33.655068 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8189a7d7-a27b-47e5-9850-ec413afb6c7a None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:33.685922 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:33.715349 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33717278-c56b-48e3-b83c-8f0a20b2c485 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:33.757761 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51c178a-9113-4efd-ba5e-5c47f9b2df1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:33.759651 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51c178a-9113-4efd-ba5e-5c47f9b2df1c None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:33.760444 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51c178a-9113-4efd-ba5e-5c47f9b2df1c None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:33.761048 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51c178a-9113-4efd-ba5e-5c47f9b2df1c None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:33.761934 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51c178a-9113-4efd-ba5e-5c47f9b2df1c None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:33.763731 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51c178a-9113-4efd-ba5e-5c47f9b2df1c None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:33.778102 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-d51c178a-9113-4efd-ba5e-5c47f9b2df1c None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 14 15:23:33.778102 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:33.778102 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 15:23:33.778102 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 15:23:33.778102 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 15:23:33.778102 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 15:23:33.778102 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 15:23:33.778102 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 15:23:33.778102 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 14 15:23:33.778102 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:23:33.778102 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 15:23:33.778102 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:23:33.778102 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:33.778102 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:33.778102 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:33.778102 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:33.779099 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:33.779099 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:33.779099 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:33.779099 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:33.779099 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:33.779099 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:33.779099 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:33.779099 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 15:23:33.779099 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 15:23:33.779099 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 15:23:33.779099 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 15:23:33.779099 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:33.779099 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:33.779099 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:33.779099 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:33.779099 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:33.780602 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:33.780602 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:33.780602 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:33.780602 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:33.780602 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:33.780602 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:33.780602 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:33.780602 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:33.780602 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:33.780602 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:33.780602 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:33.780602 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 15:23:33.780602 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 15:23:33.780602 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 14 15:23:33.780602 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:23:33.781610 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8189a7d7-a27b-47e5-9850-ec413afb6c7a None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:33.784245 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 415/813] 10.222.0.99 () {56 vars in 1129 bytes} [Thu Jul 14 15:23:33 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:33.792702 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51c178a-9113-4efd-ba5e-5c47f9b2df1c None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:33.803323 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:33.837000 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c46eec8-1ddc-451c-b1fe-a2740178209a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:33.838034 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c46eec8-1ddc-451c-b1fe-a2740178209a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:33.838519 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c46eec8-1ddc-451c-b1fe-a2740178209a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:33.839526 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c46eec8-1ddc-451c-b1fe-a2740178209a None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:33.840495 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c46eec8-1ddc-451c-b1fe-a2740178209a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:33.840988 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c46eec8-1ddc-451c-b1fe-a2740178209a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:33.843562 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c46eec8-1ddc-451c-b1fe-a2740178209a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:33.859633 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 399/814] 10.222.0.99 () {56 vars in 1148 bytes} [Thu Jul 14 15:23:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:33.865145 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c46eec8-1ddc-451c-b1fe-a2740178209a None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:33.929037 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51c178a-9113-4efd-ba5e-5c47f9b2df1c None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:33.955512 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:33.973695 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a13be23-aec5-4b88-86c5-3f653570eedc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:33.974812 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a13be23-aec5-4b88-86c5-3f653570eedc None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:33.975317 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a13be23-aec5-4b88-86c5-3f653570eedc None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:33.975835 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a13be23-aec5-4b88-86c5-3f653570eedc None admin] PATH_INFO: `/v3/services` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:33.976895 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a13be23-aec5-4b88-86c5-3f653570eedc None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:33.977449 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a13be23-aec5-4b88-86c5-3f653570eedc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:33.979062 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a13be23-aec5-4b88-86c5-3f653570eedc None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:33.979931 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c46eec8-1ddc-451c-b1fe-a2740178209a None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:33.986633 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 416/815] 10.222.0.99 () {56 vars in 1124 bytes} [Thu Jul 14 15:23:33 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:33.995104 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a13be23-aec5-4b88-86c5-3f653570eedc None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:34.027194 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5e8e45-461a-418d-97cc-a63d1b992675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:34.028179 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5e8e45-461a-418d-97cc-a63d1b992675 None admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:34.028468 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5e8e45-461a-418d-97cc-a63d1b992675 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:34.028749 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5e8e45-461a-418d-97cc-a63d1b992675 None admin] PATH_INFO: `/v3/projects/9048d1a6f00443ec92f11471babe5a56/users/ddd2ec3b3ebc4aacb322c95270374451/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:34.031740 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5e8e45-461a-418d-97cc-a63d1b992675 None admin] RBAC: Authorizing `identity:create_grant(project_id=9048d1a6f00443ec92f11471babe5a56, user_id=ddd2ec3b3ebc4aacb322c95270374451, role_id=d0f6ba1314f14d9f9c816bdb5d0c3aac)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:34.033691 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5e8e45-461a-418d-97cc-a63d1b992675 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:34.062529 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 400/816] 10.222.0.99 () {58 vars in 1439 bytes} [Thu Jul 14 15:23:33 2022] PUT /identity/v3/projects/9048d1a6f00443ec92f11471babe5a56/users/ddd2ec3b3ebc4aacb322c95270374451/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 15:23:34.101715 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:34.153422 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5690c4-b202-47da-a1b5-89f80bea9904 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:34.153868 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a13be23-aec5-4b88-86c5-3f653570eedc None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:34.154171 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5690c4-b202-47da-a1b5-89f80bea9904 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:34.154545 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5690c4-b202-47da-a1b5-89f80bea9904 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:34.156743 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 401/817] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:23:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 15:23:34.165336 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f555081b-cf92-4ec4-a235-4db229e639f3 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:34.165628 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f555081b-cf92-4ec4-a235-4db229e639f3 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:34.165914 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f555081b-cf92-4ec4-a235-4db229e639f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:34.192078 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-f555081b-cf92-4ec4-a235-4db229e639f3 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:23:34.194303 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f555081b-cf92-4ec4-a235-4db229e639f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:23:34.202309 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2425c367-0e93-4962-bead-12eadaf09a85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:34.204244 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2425c367-0e93-4962-bead-12eadaf09a85 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:34.204684 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2425c367-0e93-4962-bead-12eadaf09a85 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:34.205144 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2425c367-0e93-4962-bead-12eadaf09a85 None admin] PATH_INFO: `/v3/services` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:34.206019 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2425c367-0e93-4962-bead-12eadaf09a85 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:34.206279 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2425c367-0e93-4962-bead-12eadaf09a85 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:34.207734 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2425c367-0e93-4962-bead-12eadaf09a85 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:34.213635 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 417/818] 10.222.0.99 () {56 vars in 1124 bytes} [Thu Jul 14 15:23:33 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:34.219564 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2425c367-0e93-4962-bead-12eadaf09a85 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:34.298018 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 402/819] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:23:34 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 15:23:34.326340 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f555081b-cf92-4ec4-a235-4db229e639f3 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:34.347022 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2425c367-0e93-4962-bead-12eadaf09a85 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:34.385364 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4487c3-8acc-40e1-a2ef-caba7c59b3a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:34.386154 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4487c3-8acc-40e1-a2ef-caba7c59b3a0 None admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:34.386440 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4487c3-8acc-40e1-a2ef-caba7c59b3a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:34.386720 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4487c3-8acc-40e1-a2ef-caba7c59b3a0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:34.387470 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4487c3-8acc-40e1-a2ef-caba7c59b3a0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:34.389058 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4487c3-8acc-40e1-a2ef-caba7c59b3a0 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:34.393970 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:34.409418 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:34.416689 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 418/820] 10.222.0.99 () {60 vars in 1144 bytes} [Thu Jul 14 15:23:34 2022] POST /identity/v3/endpoints => generated 346 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:23:34.457377 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f555081b-cf92-4ec4-a235-4db229e639f3 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:34.511051 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b224b5-4837-41f0-a377-dac09cd8dd09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:34.512353 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b224b5-4837-41f0-a377-dac09cd8dd09 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:34.512814 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b224b5-4837-41f0-a377-dac09cd8dd09 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:34.513284 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b224b5-4837-41f0-a377-dac09cd8dd09 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:34.527858 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b224b5-4837-41f0-a377-dac09cd8dd09 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:23:34.537331 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-69b224b5-4837-41f0-a377-dac09cd8dd09 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:23:34.537331 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:34.537331 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:34.537331 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:34.537331 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:34.537331 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:34.537331 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:34.537331 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:34.537331 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:34.537331 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:34.537331 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:34.537331 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:34.537331 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:23:34.537331 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:23:34.537331 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:23:34.537331 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:23:34.538690 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:23:34.538690 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:23:34.538690 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:23:34.538690 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:23:34.538690 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:34.538690 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:34.538690 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:34.538690 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:34.538690 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:34.538690 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:34.538690 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:34.538690 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:34.538690 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:34.538690 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:34.538690 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:34.538690 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:34.539790 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:34.539790 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:34.539790 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:34.539790 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:34.539790 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:23:34.539790 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:23:34.539790 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:23:34.539790 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:23:34.539790 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:23:34.539790 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:23:34.539790 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:23:34.539790 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:23:34.542529 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 403/821] 10.222.0.99 () {56 vars in 1096 bytes} [Thu Jul 14 15:23:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 15:23:34.548384 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4487c3-8acc-40e1-a2ef-caba7c59b3a0 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:34.669592 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4487c3-8acc-40e1-a2ef-caba7c59b3a0 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:34.711346 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3741ddb-97e2-4b46-8b55-50f4e85b6417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:34.712029 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3741ddb-97e2-4b46-8b55-50f4e85b6417 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:34.712329 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3741ddb-97e2-4b46-8b55-50f4e85b6417 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:34.712619 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3741ddb-97e2-4b46-8b55-50f4e85b6417 None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:34.713391 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3741ddb-97e2-4b46-8b55-50f4e85b6417 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:34.713705 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3741ddb-97e2-4b46-8b55-50f4e85b6417 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:34.715119 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3741ddb-97e2-4b46-8b55-50f4e85b6417 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:34.725254 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 419/822] 10.222.0.99 () {56 vars in 1099 bytes} [Thu Jul 14 15:23:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:34.730918 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b224b5-4837-41f0-a377-dac09cd8dd09 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:34.845758 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b224b5-4837-41f0-a377-dac09cd8dd09 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:34.877020 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:34.887073 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d32948-dc76-44a0-9be7-c422f1c66f23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:34.887955 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d32948-dc76-44a0-9be7-c422f1c66f23 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:34.888247 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d32948-dc76-44a0-9be7-c422f1c66f23 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:34.888529 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d32948-dc76-44a0-9be7-c422f1c66f23 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:34.900875 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d32948-dc76-44a0-9be7-c422f1c66f23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:34.902354 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d32948-dc76-44a0-9be7-c422f1c66f23 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:34.921198 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-27d32948-dc76-44a0-9be7-c422f1c66f23 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:23:34.921198 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:34.921198 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:34.921198 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:34.921198 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:34.921198 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:34.921198 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:34.921198 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:34.921198 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:34.921198 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:34.921198 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:34.921198 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:34.921198 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:23:34.921198 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:23:34.921198 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:23:34.922603 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:23:34.922603 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:34.922603 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:34.922603 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:34.922603 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:34.922603 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:34.922603 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:34.922603 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:34.922603 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:34.922603 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:34.922603 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:34.922603 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:34.922603 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:34.922603 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:34.922603 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:34.922603 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:34.922603 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:34.923629 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:23:34.923629 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:23:34.923629 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:23:34.923629 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:23:34.923629 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 404/823] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:23:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:34.928699 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3741ddb-97e2-4b46-8b55-50f4e85b6417 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:34.990618 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:35.051877 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3741ddb-97e2-4b46-8b55-50f4e85b6417 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:35.069537 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62318e07-ed75-4685-bab3-2aaf3985c84a None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:35.069889 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62318e07-ed75-4685-bab3-2aaf3985c84a None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:35.070106 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62318e07-ed75-4685-bab3-2aaf3985c84a None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:35.072774 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 405/824] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:23:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 15:23:35.076995 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107ff83d-da7e-4ece-b2f4-878f27fd7d4d None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:35.077325 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107ff83d-da7e-4ece-b2f4-878f27fd7d4d None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:35.078270 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107ff83d-da7e-4ece-b2f4-878f27fd7d4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:35.100082 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105987ce-2f40-454c-92be-7acc37fc4832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:35.100657 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-107ff83d-da7e-4ece-b2f4-878f27fd7d4d None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:23:35.100772 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105987ce-2f40-454c-92be-7acc37fc4832 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:35.101050 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105987ce-2f40-454c-92be-7acc37fc4832 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:35.101372 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105987ce-2f40-454c-92be-7acc37fc4832 None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:35.101873 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-107ff83d-da7e-4ece-b2f4-878f27fd7d4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:23:35.102126 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105987ce-2f40-454c-92be-7acc37fc4832 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:35.102444 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105987ce-2f40-454c-92be-7acc37fc4832 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:35.104294 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105987ce-2f40-454c-92be-7acc37fc4832 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:35.123024 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 420/825] 10.222.0.99 () {56 vars in 1109 bytes} [Thu Jul 14 15:23:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:35.129375 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105987ce-2f40-454c-92be-7acc37fc4832 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:35.195726 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 406/826] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:23:35 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 15:23:35.216492 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107ff83d-da7e-4ece-b2f4-878f27fd7d4d None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:35.261387 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105987ce-2f40-454c-92be-7acc37fc4832 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:35.276994 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:35.310157 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47a8c56-668d-416d-8a64-27c4993876c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:35.311284 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47a8c56-668d-416d-8a64-27c4993876c1 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:35.311786 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47a8c56-668d-416d-8a64-27c4993876c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:35.312220 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47a8c56-668d-416d-8a64-27c4993876c1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:35.322799 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47a8c56-668d-416d-8a64-27c4993876c1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:35.325034 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47a8c56-668d-416d-8a64-27c4993876c1 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:35.345360 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107ff83d-da7e-4ece-b2f4-878f27fd7d4d None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:35.346080 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-e47a8c56-668d-416d-8a64-27c4993876c1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 14 15:23:35.346080 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:35.346080 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:35.346080 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:35.346080 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:35.346080 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:35.346080 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:35.346080 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:35.346080 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:35.346080 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:35.346080 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:35.346080 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:35.346080 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 15:23:35.346080 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 15:23:35.346080 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 15:23:35.346080 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 15:23:35.347201 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:35.347201 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:35.347201 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 15:23:35.347201 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:23:35.347201 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 15:23:35.347201 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:23:35.347201 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:35.347201 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:35.347201 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:35.347201 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:35.347201 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:35.347201 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:35.347201 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:35.347201 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:35.347201 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:35.347201 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:35.347201 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:35.347201 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:35.348274 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:35.348274 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:35.348274 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 15:23:35.348274 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 15:23:35.348274 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 15:23:35.348274 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 15:23:35.348274 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 15:23:35.348274 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 15:23:35.348274 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 14 15:23:35.348274 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:23:35.355209 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 421/827] 10.222.0.99 () {56 vars in 1093 bytes} [Thu Jul 14 15:23:35 2022] GET /identity/v3/users/nova => generated 82 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:35.362653 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47a8c56-668d-416d-8a64-27c4993876c1 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:35.364101 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:35.408500 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c1407d-76a2-404a-879b-e70b2a008b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:35.409844 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c1407d-76a2-404a-879b-e70b2a008b18 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:35.410346 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c1407d-76a2-404a-879b-e70b2a008b18 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:35.410857 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c1407d-76a2-404a-879b-e70b2a008b18 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:35.423432 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c1407d-76a2-404a-879b-e70b2a008b18 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:23:35.431900 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-24c1407d-76a2-404a-879b-e70b2a008b18 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 15:23:35.431900 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:35.431900 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:35.431900 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:35.431900 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:35.431900 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:35.431900 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:35.431900 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:35.431900 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:35.431900 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:35.431900 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:35.431900 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:35.431900 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:23:35.431900 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:23:35.431900 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:23:35.431900 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:23:35.432851 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:23:35.432851 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:23:35.432851 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:23:35.432851 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:23:35.432851 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:35.432851 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:35.432851 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:35.432851 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:35.432851 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:35.432851 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:35.432851 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:35.432851 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:35.432851 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:35.432851 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:35.432851 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:35.432851 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:35.433682 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:35.433682 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:35.433682 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:35.433682 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:35.433682 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:23:35.433682 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:23:35.433682 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:23:35.433682 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:23:35.433682 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:23:35.433682 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:23:35.433682 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 15:23:35.433682 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:23:35.436516 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 407/828] 10.222.0.99 () {56 vars in 1102 bytes} [Thu Jul 14 15:23:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 15:23:35.444191 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c1407d-76a2-404a-879b-e70b2a008b18 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:35.490585 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47a8c56-668d-416d-8a64-27c4993876c1 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:35.533953 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbb19b7-0ab2-42ec-8dba-20fc15ff6865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:35.534423 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbb19b7-0ab2-42ec-8dba-20fc15ff6865 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:35.534715 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbb19b7-0ab2-42ec-8dba-20fc15ff6865 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:35.534995 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbb19b7-0ab2-42ec-8dba-20fc15ff6865 None admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:35.535972 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbb19b7-0ab2-42ec-8dba-20fc15ff6865 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:35.536431 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbb19b7-0ab2-42ec-8dba-20fc15ff6865 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:35.538014 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbb19b7-0ab2-42ec-8dba-20fc15ff6865 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:35.559125 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c1407d-76a2-404a-879b-e70b2a008b18 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:35.569323 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 422/829] 10.222.0.99 () {56 vars in 1133 bytes} [Thu Jul 14 15:23:35 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:35.572649 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbb19b7-0ab2-42ec-8dba-20fc15ff6865 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:35.610607 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:35.613094 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d72e54c-5887-4b00-b6ef-4679d08f2409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:35.614131 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d72e54c-5887-4b00-b6ef-4679d08f2409 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:35.614521 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d72e54c-5887-4b00-b6ef-4679d08f2409 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:35.614902 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d72e54c-5887-4b00-b6ef-4679d08f2409 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:35.615800 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d72e54c-5887-4b00-b6ef-4679d08f2409 None admin] RBAC: Adding query filter params (name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:35.616212 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d72e54c-5887-4b00-b6ef-4679d08f2409 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:35.619486 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d72e54c-5887-4b00-b6ef-4679d08f2409 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:35.635080 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 408/830] 10.222.0.99 () {56 vars in 1103 bytes} [Thu Jul 14 15:23:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:23:35.637311 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d72e54c-5887-4b00-b6ef-4679d08f2409 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:35.675997 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:35.709437 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbb19b7-0ab2-42ec-8dba-20fc15ff6865 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:35.756176 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c31133-5e30-489e-abdc-bdd84f6a445d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:35.757125 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c31133-5e30-489e-abdc-bdd84f6a445d None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:35.757576 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c31133-5e30-489e-abdc-bdd84f6a445d None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:35.758005 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c31133-5e30-489e-abdc-bdd84f6a445d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:35.773299 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d72e54c-5887-4b00-b6ef-4679d08f2409 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:35.819754 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431c6a96-964b-4e99-b35e-b50227566224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:35.820813 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431c6a96-964b-4e99-b35e-b50227566224 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:35.821359 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431c6a96-964b-4e99-b35e-b50227566224 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:35.821790 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431c6a96-964b-4e99-b35e-b50227566224 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:35.834486 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431c6a96-964b-4e99-b35e-b50227566224 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:35.836404 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431c6a96-964b-4e99-b35e-b50227566224 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:35.843585 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c31133-5e30-489e-abdc-bdd84f6a445d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:35.845430 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c31133-5e30-489e-abdc-bdd84f6a445d None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:35.856322 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-431c6a96-964b-4e99-b35e-b50227566224 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:23:35.856322 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:35.856322 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:35.856322 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:35.856322 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:35.856322 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:35.856322 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:35.856322 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:35.856322 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:35.856322 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:35.856322 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:35.856322 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:35.856322 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:23:35.856322 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:23:35.856322 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:23:35.857518 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:23:35.857518 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:35.857518 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:35.857518 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:35.857518 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:35.857518 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:35.857518 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:35.857518 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:35.857518 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:35.857518 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:35.857518 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:35.857518 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:35.857518 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:35.857518 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:35.857518 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:35.857518 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:35.857518 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:35.858400 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:23:35.858400 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:23:35.858400 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:23:35.858400 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:23:35.863331 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 409/831] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:23:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:35.871017 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431c6a96-964b-4e99-b35e-b50227566224 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:35.882836 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:35.965779 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 423/832] 10.222.0.99 () {58 vars in 1282 bytes} [Thu Jul 14 15:23:35 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 15:23:35.969223 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c31133-5e30-489e-abdc-bdd84f6a445d None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:35.973641 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:35.998024 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431c6a96-964b-4e99-b35e-b50227566224 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:36.035046 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b42c80-0843-49ad-b3e5-018bca070562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:36.036139 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b42c80-0843-49ad-b3e5-018bca070562 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:36.036689 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b42c80-0843-49ad-b3e5-018bca070562 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:36.037202 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b42c80-0843-49ad-b3e5-018bca070562 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:36.038150 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b42c80-0843-49ad-b3e5-018bca070562 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:36.038668 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b42c80-0843-49ad-b3e5-018bca070562 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:36.040266 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b42c80-0843-49ad-b3e5-018bca070562 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:36.057291 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 410/833] 10.222.0.99 () {56 vars in 1109 bytes} [Thu Jul 14 15:23:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:23:36.058480 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b42c80-0843-49ad-b3e5-018bca070562 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:36.087510 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c31133-5e30-489e-abdc-bdd84f6a445d None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:36.120808 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:36.129176 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87085d2a-b82c-462c-b485-951b579fa969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:36.130222 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87085d2a-b82c-462c-b485-951b579fa969 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:36.130337 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87085d2a-b82c-462c-b485-951b579fa969 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:36.130589 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87085d2a-b82c-462c-b485-951b579fa969 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:36.143288 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87085d2a-b82c-462c-b485-951b579fa969 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:36.144780 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87085d2a-b82c-462c-b485-951b579fa969 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:36.164475 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-87085d2a-b82c-462c-b485-951b579fa969 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:23:36.164475 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:36.164475 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:36.164475 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:36.164475 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:36.164475 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:36.164475 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:36.164475 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:36.164475 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:36.164475 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:36.164475 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:36.164475 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:36.164475 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:23:36.164475 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:23:36.164475 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:23:36.165438 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:23:36.165438 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:36.165438 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:36.165438 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:36.165438 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:36.165438 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:36.165438 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:36.165438 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:36.165438 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:36.165438 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:36.165438 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:36.165438 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:36.165438 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:36.165438 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:36.165438 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:36.165438 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:36.165438 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:36.166314 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:23:36.166314 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:23:36.166314 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:23:36.166314 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:23:36.169529 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b42c80-0843-49ad-b3e5-018bca070562 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:36.170498 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 424/834] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:23:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:36.172553 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87085d2a-b82c-462c-b485-951b579fa969 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:36.207229 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d4a9ff-0618-44bc-8699-10ad1c653557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:36.208425 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d4a9ff-0618-44bc-8699-10ad1c653557 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:36.208982 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d4a9ff-0618-44bc-8699-10ad1c653557 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:36.209470 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d4a9ff-0618-44bc-8699-10ad1c653557 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:36.217413 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d4a9ff-0618-44bc-8699-10ad1c653557 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:36.219937 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d4a9ff-0618-44bc-8699-10ad1c653557 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:36.235850 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-69d4a9ff-0618-44bc-8699-10ad1c653557 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 15:23:36.235850 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:36.235850 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:36.235850 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:36.235850 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:36.235850 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:36.235850 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:36.235850 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:36.235850 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:36.235850 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:36.235850 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:36.235850 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:36.235850 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 15:23:36.235850 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 15:23:36.235850 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 15:23:36.235850 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 15:23:36.236827 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:36.236827 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:36.236827 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 15:23:36.236827 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:23:36.236827 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 15:23:36.236827 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:23:36.236827 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:36.236827 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:36.236827 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:36.236827 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:36.236827 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:36.236827 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:36.236827 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:36.236827 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:36.236827 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:36.236827 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:36.236827 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:36.236827 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:36.237756 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:36.237756 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:36.237756 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 15:23:36.237756 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 15:23:36.237756 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 15:23:36.237756 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 15:23:36.237756 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 15:23:36.237756 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 15:23:36.237756 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 15:23:36.237756 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:23:36.244855 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:36.245127 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 411/835] 10.222.0.99 () {56 vars in 1117 bytes} [Thu Jul 14 15:23:36 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:36.246218 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d4a9ff-0618-44bc-8699-10ad1c653557 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:36.293956 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87085d2a-b82c-462c-b485-951b579fa969 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:36.337402 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b357c1-758a-4243-917a-19b01ff13f2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:36.338054 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b357c1-758a-4243-917a-19b01ff13f2c None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:36.338290 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b357c1-758a-4243-917a-19b01ff13f2c None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:36.338576 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b357c1-758a-4243-917a-19b01ff13f2c None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:36.339379 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b357c1-758a-4243-917a-19b01ff13f2c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:36.339724 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b357c1-758a-4243-917a-19b01ff13f2c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:36.341279 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b357c1-758a-4243-917a-19b01ff13f2c None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:36.360163 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 425/836] 10.222.0.99 () {56 vars in 1109 bytes} [Thu Jul 14 15:23:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:36.363424 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b357c1-758a-4243-917a-19b01ff13f2c None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:36.365242 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d4a9ff-0618-44bc-8699-10ad1c653557 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:36.408064 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d85133-a1e2-4d28-b08b-e5c6853bb5a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:36.409148 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d85133-a1e2-4d28-b08b-e5c6853bb5a8 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:36.409436 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d85133-a1e2-4d28-b08b-e5c6853bb5a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:36.409722 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d85133-a1e2-4d28-b08b-e5c6853bb5a8 None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:36.410576 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d85133-a1e2-4d28-b08b-e5c6853bb5a8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:36.410904 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d85133-a1e2-4d28-b08b-e5c6853bb5a8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:36.412376 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d85133-a1e2-4d28-b08b-e5c6853bb5a8 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:36.421878 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:36.435326 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 412/837] 10.222.0.99 () {56 vars in 1149 bytes} [Thu Jul 14 15:23:36 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:36.436139 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe97219-d9d2-4854-94a2-efd4891b87b6 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:36.436272 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe97219-d9d2-4854-94a2-efd4891b87b6 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:36.436491 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe97219-d9d2-4854-94a2-efd4891b87b6 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:36.438442 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 413/838] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:23:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:23:36.444155 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2dbfba-7664-4f52-b5b7-5131a3b73101 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:36.444459 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2dbfba-7664-4f52-b5b7-5131a3b73101 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:36.444681 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2dbfba-7664-4f52-b5b7-5131a3b73101 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:36.465796 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-2e2dbfba-7664-4f52-b5b7-5131a3b73101 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:23:36.468601 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-2e2dbfba-7664-4f52-b5b7-5131a3b73101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:23:36.511568 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b357c1-758a-4243-917a-19b01ff13f2c None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:36.545864 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:36.559205 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed178ab-135f-4a5b-91f3-d49fdbc35169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:36.560284 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed178ab-135f-4a5b-91f3-d49fdbc35169 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:36.560680 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed178ab-135f-4a5b-91f3-d49fdbc35169 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:36.560955 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed178ab-135f-4a5b-91f3-d49fdbc35169 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:36.567844 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 414/839] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:23:36 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 15:23:36.567844 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2dbfba-7664-4f52-b5b7-5131a3b73101 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:36.573643 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed178ab-135f-4a5b-91f3-d49fdbc35169 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:36.575117 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed178ab-135f-4a5b-91f3-d49fdbc35169 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:36.591357 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-bed178ab-135f-4a5b-91f3-d49fdbc35169 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:23:36.591357 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:36.591357 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:36.591357 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:36.591357 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:36.591357 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:36.591357 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:36.591357 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:36.591357 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:36.591357 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:36.591357 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:36.591357 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:36.591357 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 15:23:36.591357 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 15:23:36.591357 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 15:23:36.592323 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 15:23:36.592323 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:36.592323 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:36.592323 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:36.592323 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:36.592323 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:36.592323 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:36.592323 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:36.592323 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:36.592323 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:36.592323 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:36.592323 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:36.592323 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:36.592323 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:36.592323 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:36.592323 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:36.592323 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:36.593747 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 15:23:36.593747 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 15:23:36.593747 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 15:23:36.593747 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 15:23:36.593747 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 15:23:36.593747 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 15:23:36.593747 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:23:36.593747 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:23:36.593747 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 426/840] 10.222.0.99 () {56 vars in 1111 bytes} [Thu Jul 14 15:23:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:36.594590 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed178ab-135f-4a5b-91f3-d49fdbc35169 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:36.688765 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2dbfba-7664-4f52-b5b7-5131a3b73101 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:36.705688 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed178ab-135f-4a5b-91f3-d49fdbc35169 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:36.707292 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:36.729688 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28feb1c-f31f-485a-b6f2-12cc6c323d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:36.730343 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28feb1c-f31f-485a-b6f2-12cc6c323d67 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:36.730631 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28feb1c-f31f-485a-b6f2-12cc6c323d67 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:36.730913 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28feb1c-f31f-485a-b6f2-12cc6c323d67 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:36.745190 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c81e110-b1ee-47e4-94be-276850c5343c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:36.745987 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c81e110-b1ee-47e4-94be-276850c5343c None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:36.746269 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c81e110-b1ee-47e4-94be-276850c5343c None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:36.746549 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c81e110-b1ee-47e4-94be-276850c5343c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:36.756873 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c81e110-b1ee-47e4-94be-276850c5343c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:23:36.764696 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-1c81e110-b1ee-47e4-94be-276850c5343c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:23:36.764696 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:36.764696 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:36.764696 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:36.764696 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:36.764696 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:36.764696 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:36.764696 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:36.764696 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:36.764696 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:36.764696 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:36.764696 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:36.764696 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:23:36.764696 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:23:36.764696 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:23:36.764696 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:23:36.766372 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:23:36.766372 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:23:36.766372 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:23:36.766372 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:23:36.766372 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:36.766372 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:36.766372 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:36.766372 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:36.766372 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:36.766372 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:36.766372 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:36.766372 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:36.766372 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:36.766372 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:36.766372 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:36.766372 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:36.767264 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:36.767264 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:36.767264 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:36.767264 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:36.767264 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:23:36.767264 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:23:36.767264 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:23:36.767264 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:23:36.767264 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:23:36.767264 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:23:36.767264 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:23:36.767264 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:23:36.767264 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 427/841] 10.222.0.99 () {56 vars in 1096 bytes} [Thu Jul 14 15:23:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:36.768186 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c81e110-b1ee-47e4-94be-276850c5343c None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:36.794980 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:36.808708 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28feb1c-f31f-485a-b6f2-12cc6c323d67 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:36.810201 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28feb1c-f31f-485a-b6f2-12cc6c323d67 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:36.892577 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c81e110-b1ee-47e4-94be-276850c5343c None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:36.928292 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 415/842] 10.222.0.99 () {58 vars in 1282 bytes} [Thu Jul 14 15:23:36 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 15:23:36.929658 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0df4f6f-2f0c-4b91-ae69-d4338b48c7be None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:36.929958 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0df4f6f-2f0c-4b91-ae69-d4338b48c7be None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:36.930241 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0df4f6f-2f0c-4b91-ae69-d4338b48c7be None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:36.931537 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 416/843] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:23:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:23:36.932264 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0df4f6f-2f0c-4b91-ae69-d4338b48c7be None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:36.937298 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcf3b5f-0960-4613-b075-d6b14e3155e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:36.938182 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcf3b5f-0960-4613-b075-d6b14e3155e4 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:36.938561 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcf3b5f-0960-4613-b075-d6b14e3155e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:36.938953 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcf3b5f-0960-4613-b075-d6b14e3155e4 None admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:36.939896 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcf3b5f-0960-4613-b075-d6b14e3155e4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:36.940298 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcf3b5f-0960-4613-b075-d6b14e3155e4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:36.941848 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcf3b5f-0960-4613-b075-d6b14e3155e4 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:36.955751 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 428/844] 10.222.0.99 () {56 vars in 1148 bytes} [Thu Jul 14 15:23:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:36.956615 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcf3b5f-0960-4613-b075-d6b14e3155e4 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:36.991270 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:37.045469 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:37.060029 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0df4f6f-2f0c-4b91-ae69-d4338b48c7be None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:37.078105 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcf3b5f-0960-4613-b075-d6b14e3155e4 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:37.097997 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2736bd6a-071d-4d04-89ca-32c399271e36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:37.098670 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2736bd6a-071d-4d04-89ca-32c399271e36 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:37.098958 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2736bd6a-071d-4d04-89ca-32c399271e36 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:37.099794 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2736bd6a-071d-4d04-89ca-32c399271e36 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:37.100567 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2736bd6a-071d-4d04-89ca-32c399271e36 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:37.100880 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2736bd6a-071d-4d04-89ca-32c399271e36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:37.102273 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2736bd6a-071d-4d04-89ca-32c399271e36 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:37.111985 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 417/845] 10.222.0.99 () {56 vars in 1099 bytes} [Thu Jul 14 15:23:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:37.113284 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc1b941-4081-4718-b27a-5ef079854c27 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:37.113581 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc1b941-4081-4718-b27a-5ef079854c27 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:37.113866 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc1b941-4081-4718-b27a-5ef079854c27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:37.119899 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a463d1-d087-458e-93c4-95a29451b3e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:37.120738 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a463d1-d087-458e-93c4-95a29451b3e3 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:37.120990 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a463d1-d087-458e-93c4-95a29451b3e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:37.121274 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a463d1-d087-458e-93c4-95a29451b3e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:37.138074 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a463d1-d087-458e-93c4-95a29451b3e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:37.139563 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a463d1-d087-458e-93c4-95a29451b3e3 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:37.146472 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-2dc1b941-4081-4718-b27a-5ef079854c27 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:23:37.147669 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-2dc1b941-4081-4718-b27a-5ef079854c27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:23:37.159193 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-f1a463d1-d087-458e-93c4-95a29451b3e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:23:37.159193 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:37.159193 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:37.159193 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:37.159193 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:37.159193 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:37.159193 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:37.159193 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:37.159193 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:37.159193 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:37.159193 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:37.159193 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:37.159193 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:23:37.159193 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:23:37.159193 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:23:37.160150 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:23:37.160150 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:37.160150 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:37.160150 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:37.160150 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:37.160150 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:37.160150 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:37.160150 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:37.160150 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:37.160150 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:37.160150 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:37.160150 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:37.160150 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:37.160150 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:37.160150 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:37.160150 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:37.160150 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:37.161057 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:23:37.161057 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:23:37.161057 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:23:37.161057 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:23:37.161057 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 429/846] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:23:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:37.161333 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a463d1-d087-458e-93c4-95a29451b3e3 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:37.224502 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 418/847] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:23:37 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 15:23:37.225301 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc1b941-4081-4718-b27a-5ef079854c27 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:37.242359 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:37.270803 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a463d1-d087-458e-93c4-95a29451b3e3 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:37.305099 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b956e0ef-cf79-4200-b8af-985e8a274ad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:37.305765 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b956e0ef-cf79-4200-b8af-985e8a274ad1 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:37.306055 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b956e0ef-cf79-4200-b8af-985e8a274ad1 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:37.306336 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b956e0ef-cf79-4200-b8af-985e8a274ad1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:37.307178 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b956e0ef-cf79-4200-b8af-985e8a274ad1 None admin] RBAC: Adding query filter params (role.id=d0f6ba1314f14d9f9c816bdb5d0c3aac, scope.project.id=9048d1a6f00443ec92f11471babe5a56, user.id=3a9b1c2622934c5185f3b5f1ea623bc0) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:37.307497 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b956e0ef-cf79-4200-b8af-985e8a274ad1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:37.308974 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b956e0ef-cf79-4200-b8af-985e8a274ad1 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:37.315739 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 430/848] 10.222.0.99 () {56 vars in 1374 bytes} [Thu Jul 14 15:23:37 2022] GET /identity/v3/role_assignments?user.id=3a9b1c2622934c5185f3b5f1ea623bc0&scope.project.id=9048d1a6f00443ec92f11471babe5a56&role.id=d0f6ba1314f14d9f9c816bdb5d0c3aac => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:37.316446 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b956e0ef-cf79-4200-b8af-985e8a274ad1 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:37.326142 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:37.351678 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc1b941-4081-4718-b27a-5ef079854c27 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:37.395753 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53c9246-7414-439d-bb07-799efa959b75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:37.396743 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53c9246-7414-439d-bb07-799efa959b75 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:37.396974 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53c9246-7414-439d-bb07-799efa959b75 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:37.397658 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53c9246-7414-439d-bb07-799efa959b75 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:37.410641 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53c9246-7414-439d-bb07-799efa959b75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:37.412426 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53c9246-7414-439d-bb07-799efa959b75 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:37.434193 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b956e0ef-cf79-4200-b8af-985e8a274ad1 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:37.434401 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-c53c9246-7414-439d-bb07-799efa959b75 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:23:37.434401 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:37.434401 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:37.434401 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:37.434401 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:37.434401 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:37.434401 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:37.434401 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:37.434401 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:37.434401 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:37.434401 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:37.434401 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:37.434401 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:23:37.434401 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:23:37.434401 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:23:37.435252 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:23:37.435252 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:37.435252 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:37.435252 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:37.435252 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:37.435252 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:37.435252 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:37.435252 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:37.435252 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:37.435252 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:37.435252 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:37.435252 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:37.435252 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:37.435252 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:37.435252 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:37.435252 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:37.435252 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:37.436174 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:23:37.436174 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:23:37.436174 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:23:37.436174 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:23:37.440032 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 419/849] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:23:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:37.441301 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53c9246-7414-439d-bb07-799efa959b75 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:37.478154 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0f72d3-dea8-4c03-9cd5-a1cd1214e9a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:37.479080 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0f72d3-dea8-4c03-9cd5-a1cd1214e9a7 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:37.479080 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0f72d3-dea8-4c03-9cd5-a1cd1214e9a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:37.479378 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0f72d3-dea8-4c03-9cd5-a1cd1214e9a7 None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:37.480254 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0f72d3-dea8-4c03-9cd5-a1cd1214e9a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:37.480446 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0f72d3-dea8-4c03-9cd5-a1cd1214e9a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:37.481826 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0f72d3-dea8-4c03-9cd5-a1cd1214e9a7 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:37.492150 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:37.499979 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 431/850] 10.222.0.99 () {56 vars in 1109 bytes} [Thu Jul 14 15:23:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:37.501336 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0f72d3-dea8-4c03-9cd5-a1cd1214e9a7 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:37.573041 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53c9246-7414-439d-bb07-799efa959b75 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:37.611129 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:37.617418 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0f72d3-dea8-4c03-9cd5-a1cd1214e9a7 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:37.618027 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057799c9-6a36-40ec-a570-3ff77b0770d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:37.618844 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057799c9-6a36-40ec-a570-3ff77b0770d6 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:37.619131 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057799c9-6a36-40ec-a570-3ff77b0770d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:37.619409 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057799c9-6a36-40ec-a570-3ff77b0770d6 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:37.620108 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057799c9-6a36-40ec-a570-3ff77b0770d6 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:37.621487 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057799c9-6a36-40ec-a570-3ff77b0770d6 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:37.634654 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-057799c9-6a36-40ec-a570-3ff77b0770d6 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 14 15:23:37.634654 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:37.634654 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 15:23:37.634654 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 15:23:37.634654 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 15:23:37.634654 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 15:23:37.634654 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 15:23:37.634654 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 15:23:37.634654 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 14 15:23:37.634654 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:23:37.634654 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 15:23:37.634654 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:23:37.634654 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:37.634654 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:37.634654 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:37.634654 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:37.634654 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:37.636096 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:37.636096 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:37.636096 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:37.636096 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:37.636096 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:37.636096 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:37.636096 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 15:23:37.636096 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 15:23:37.636096 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 15:23:37.636096 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 15:23:37.636096 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:37.636096 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:37.636096 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:37.636096 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:37.636096 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:37.636096 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:37.636937 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:37.636937 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:37.636937 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:37.636937 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:37.636937 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:37.636937 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:37.636937 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:37.636937 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:37.636937 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:37.636937 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:37.636937 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 15:23:37.636937 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 15:23:37.636937 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 14 15:23:37.636937 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:23:37.636937 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 420/851] 10.222.0.99 () {56 vars in 1114 bytes} [Thu Jul 14 15:23:37 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:37.637738 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057799c9-6a36-40ec-a570-3ff77b0770d6 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:37.661685 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdea218-480c-47b4-8c85-5a6ffcbfaed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:37.662285 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdea218-480c-47b4-8c85-5a6ffcbfaed6 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:37.662577 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdea218-480c-47b4-8c85-5a6ffcbfaed6 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:37.662860 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdea218-480c-47b4-8c85-5a6ffcbfaed6 None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:37.663617 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdea218-480c-47b4-8c85-5a6ffcbfaed6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:37.663936 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdea218-480c-47b4-8c85-5a6ffcbfaed6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:37.665332 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdea218-480c-47b4-8c85-5a6ffcbfaed6 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:37.683183 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 432/852] 10.222.0.99 () {56 vars in 1109 bytes} [Thu Jul 14 15:23:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:37.683183 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdea218-480c-47b4-8c85-5a6ffcbfaed6 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:37.746195 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:37.759547 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057799c9-6a36-40ec-a570-3ff77b0770d6 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:37.794166 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdea218-480c-47b4-8c85-5a6ffcbfaed6 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:37.797024 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008d95d8-e05e-4126-9016-a16c86824cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:37.797825 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008d95d8-e05e-4126-9016-a16c86824cdc None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:37.798115 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008d95d8-e05e-4126-9016-a16c86824cdc None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:37.798400 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008d95d8-e05e-4126-9016-a16c86824cdc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:37.805975 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008d95d8-e05e-4126-9016-a16c86824cdc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:37.807419 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008d95d8-e05e-4126-9016-a16c86824cdc None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:37.823144 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-008d95d8-e05e-4126-9016-a16c86824cdc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:23:37.823144 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:37.823144 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:37.823144 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:37.823144 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:37.823144 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:37.823144 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:37.823144 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:37.823144 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:37.823144 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:37.823144 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:37.823144 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:37.823144 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 15:23:37.823144 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 15:23:37.823144 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 15:23:37.825049 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 15:23:37.825049 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:37.825049 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:37.825049 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:37.825049 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:37.825049 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:37.825049 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:37.825049 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:37.825049 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:37.825049 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:37.825049 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:37.825049 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:37.825049 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:37.825049 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:37.825049 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:37.825049 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:37.825049 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:37.825928 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 15:23:37.825928 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 15:23:37.825928 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 15:23:37.825928 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 15:23:37.825928 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 15:23:37.825928 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 15:23:37.825928 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:23:37.825928 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:23:37.830298 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 421/853] 10.222.0.99 () {56 vars in 1111 bytes} [Thu Jul 14 15:23:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:37.831204 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008d95d8-e05e-4126-9016-a16c86824cdc None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:37.833920 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daff376-597a-496a-bb12-dffd021f06a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:37.834607 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daff376-597a-496a-bb12-dffd021f06a1 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:37.834891 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daff376-597a-496a-bb12-dffd021f06a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:37.835173 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daff376-597a-496a-bb12-dffd021f06a1 None admin] PATH_INFO: `/v3/services` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:37.835937 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daff376-597a-496a-bb12-dffd021f06a1 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:37.836248 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daff376-597a-496a-bb12-dffd021f06a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:37.837630 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daff376-597a-496a-bb12-dffd021f06a1 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:37.842400 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 433/854] 10.222.0.99 () {56 vars in 1114 bytes} [Thu Jul 14 15:23:37 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:37.843902 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daff376-597a-496a-bb12-dffd021f06a1 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:37.899664 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:37.953134 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008d95d8-e05e-4126-9016-a16c86824cdc None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:37.956750 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daff376-597a-496a-bb12-dffd021f06a1 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:37.979945 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:37.995314 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f770736-6aaf-4e34-86d8-cbcfb33274d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:37.996144 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f770736-6aaf-4e34-86d8-cbcfb33274d9 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:37.996429 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f770736-6aaf-4e34-86d8-cbcfb33274d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:37.996714 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f770736-6aaf-4e34-86d8-cbcfb33274d9 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:38.003261 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bb56b9-0b24-46f4-ae5c-0fd6f1966ebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:38.004131 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bb56b9-0b24-46f4-ae5c-0fd6f1966ebc None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:38.004419 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bb56b9-0b24-46f4-ae5c-0fd6f1966ebc None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:38.004517 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f770736-6aaf-4e34-86d8-cbcfb33274d9 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:38.004836 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bb56b9-0b24-46f4-ae5c-0fd6f1966ebc None admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:38.005644 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bb56b9-0b24-46f4-ae5c-0fd6f1966ebc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:38.005955 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bb56b9-0b24-46f4-ae5c-0fd6f1966ebc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:38.006116 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f770736-6aaf-4e34-86d8-cbcfb33274d9 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:38.007380 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bb56b9-0b24-46f4-ae5c-0fd6f1966ebc None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:38.020651 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 434/855] 10.222.0.99 () {56 vars in 1148 bytes} [Thu Jul 14 15:23:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:38.021314 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bb56b9-0b24-46f4-ae5c-0fd6f1966ebc None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:38.022525 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-6f770736-6aaf-4e34-86d8-cbcfb33274d9 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 14 15:23:38.022525 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:38.022525 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:38.022525 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:38.022525 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:38.022525 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:38.022525 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:38.022525 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:38.022525 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:38.022525 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:38.022525 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:38.022525 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:38.022525 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 15:23:38.022525 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 15:23:38.022525 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 15:23:38.022525 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 15:23:38.023449 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:38.023449 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:38.023449 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 15:23:38.023449 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:23:38.023449 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 15:23:38.023449 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:23:38.023449 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:38.023449 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:38.023449 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:38.023449 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:38.023449 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:38.023449 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:38.023449 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:38.023449 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:38.023449 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:38.023449 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:38.023449 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:38.023449 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:38.024433 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:38.024433 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:38.024433 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 15:23:38.024433 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 15:23:38.024433 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 15:23:38.024433 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 15:23:38.024433 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 15:23:38.024433 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 15:23:38.024433 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 14 15:23:38.024433 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:23:38.028137 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 422/856] 10.222.0.99 () {56 vars in 1096 bytes} [Thu Jul 14 15:23:37 2022] GET /identity/v3/users/swift => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:38.029242 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f770736-6aaf-4e34-86d8-cbcfb33274d9 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:38.130894 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bb56b9-0b24-46f4-ae5c-0fd6f1966ebc None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:38.137585 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f770736-6aaf-4e34-86d8-cbcfb33274d9 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:38.155090 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:38.169787 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9620556-e0d2-4c2f-937b-8206c3ba3d98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:38.170455 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9620556-e0d2-4c2f-937b-8206c3ba3d98 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:38.170741 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9620556-e0d2-4c2f-937b-8206c3ba3d98 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:38.171571 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9620556-e0d2-4c2f-937b-8206c3ba3d98 None admin] PATH_INFO: `/v3/services` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:38.172568 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9620556-e0d2-4c2f-937b-8206c3ba3d98 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:38.172879 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9620556-e0d2-4c2f-937b-8206c3ba3d98 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:38.174245 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9620556-e0d2-4c2f-937b-8206c3ba3d98 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:38.174536 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbe4d98-bb77-4902-86f6-51b66c852b9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:38.175190 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbe4d98-bb77-4902-86f6-51b66c852b9a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:38.175483 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbe4d98-bb77-4902-86f6-51b66c852b9a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:38.175793 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbe4d98-bb77-4902-86f6-51b66c852b9a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:38.176661 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbe4d98-bb77-4902-86f6-51b66c852b9a None admin] RBAC: Adding query filter params (role.id=74e3b597938f49d095cb424c54b5a811, scope.project.id=9048d1a6f00443ec92f11471babe5a56, user.id=adaeadf613954d7393b3fd3bae68c362) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:38.176977 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbe4d98-bb77-4902-86f6-51b66c852b9a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:38.178545 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbe4d98-bb77-4902-86f6-51b66c852b9a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:38.183692 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 435/857] 10.222.0.99 () {56 vars in 1114 bytes} [Thu Jul 14 15:23:38 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:38.183692 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9620556-e0d2-4c2f-937b-8206c3ba3d98 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:38.196876 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 423/858] 10.222.0.99 () {56 vars in 1374 bytes} [Thu Jul 14 15:23:38 2022] GET /identity/v3/role_assignments?user.id=adaeadf613954d7393b3fd3bae68c362&scope.project.id=9048d1a6f00443ec92f11471babe5a56&role.id=74e3b597938f49d095cb424c54b5a811 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:38.250613 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:38.352598 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9620556-e0d2-4c2f-937b-8206c3ba3d98 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:38.395153 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df4c81f-d3e3-43a5-847e-3b8e96339ea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:38.396269 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df4c81f-d3e3-43a5-847e-3b8e96339ea4 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:38.396778 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df4c81f-d3e3-43a5-847e-3b8e96339ea4 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:38.397268 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df4c81f-d3e3-43a5-847e-3b8e96339ea4 None admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:38.398334 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df4c81f-d3e3-43a5-847e-3b8e96339ea4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:38.398870 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df4c81f-d3e3-43a5-847e-3b8e96339ea4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:38.400562 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df4c81f-d3e3-43a5-847e-3b8e96339ea4 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:38.421806 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 436/859] 10.222.0.99 () {56 vars in 1135 bytes} [Thu Jul 14 15:23:38 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:38.428593 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbe4d98-bb77-4902-86f6-51b66c852b9a None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:38.569717 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbe4d98-bb77-4902-86f6-51b66c852b9a None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:38.607550 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce594694-3f78-4444-969c-0986b6fe3865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:38.608831 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce594694-3f78-4444-969c-0986b6fe3865 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:38.609395 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce594694-3f78-4444-969c-0986b6fe3865 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:38.609908 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce594694-3f78-4444-969c-0986b6fe3865 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:38.668426 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:38.690813 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce594694-3f78-4444-969c-0986b6fe3865 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:38.692643 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce594694-3f78-4444-969c-0986b6fe3865 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:38.933426 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 424/860] 10.222.0.99 () {58 vars in 1282 bytes} [Thu Jul 14 15:23:38 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 502 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 15:23:38.935185 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df4c81f-d3e3-43a5-847e-3b8e96339ea4 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:39.046029 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df4c81f-d3e3-43a5-847e-3b8e96339ea4 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:39.048843 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:39.083263 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b7e708-d27e-42e2-9585-4797f5a35c05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:39.084122 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b7e708-d27e-42e2-9585-4797f5a35c05 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:39.084466 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b7e708-d27e-42e2-9585-4797f5a35c05 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:39.085353 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b7e708-d27e-42e2-9585-4797f5a35c05 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:39.097243 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b7e708-d27e-42e2-9585-4797f5a35c05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:39.098728 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b7e708-d27e-42e2-9585-4797f5a35c05 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:39.118276 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-20b7e708-d27e-42e2-9585-4797f5a35c05 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:23:39.118276 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:39.118276 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:39.118276 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:39.118276 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:39.118276 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:39.118276 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:39.118276 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:39.118276 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:39.118276 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:39.118276 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:39.118276 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:39.118276 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:23:39.118276 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:23:39.118276 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:23:39.120616 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:23:39.120616 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:39.120616 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:39.120616 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:39.120616 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:39.120616 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:39.120616 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:39.120616 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:39.120616 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:39.120616 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:39.120616 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:39.120616 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:39.120616 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:39.120616 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:39.120616 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:39.120616 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:39.120616 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:39.122758 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:23:39.122758 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:23:39.122758 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:23:39.122758 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:23:39.122758 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 437/861] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:23:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 15:23:39.130597 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce594694-3f78-4444-969c-0986b6fe3865 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:39.236105 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:39.270210 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce594694-3f78-4444-969c-0986b6fe3865 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:39.328699 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71db336f-2887-482f-a4a1-5adac04ead60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:39.330219 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71db336f-2887-482f-a4a1-5adac04ead60 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:39.330908 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71db336f-2887-482f-a4a1-5adac04ead60 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:39.331627 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71db336f-2887-482f-a4a1-5adac04ead60 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:39.333178 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71db336f-2887-482f-a4a1-5adac04ead60 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:39.333925 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71db336f-2887-482f-a4a1-5adac04ead60 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:39.336674 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71db336f-2887-482f-a4a1-5adac04ead60 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:39.361120 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 425/862] 10.222.0.99 () {56 vars in 1109 bytes} [Thu Jul 14 15:23:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:39.371636 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b7e708-d27e-42e2-9585-4797f5a35c05 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:39.469154 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b7e708-d27e-42e2-9585-4797f5a35c05 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:39.509152 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b0df1a-adbc-42cb-b178-09ef0801fa10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:39.509869 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b0df1a-adbc-42cb-b178-09ef0801fa10 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:39.510159 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b0df1a-adbc-42cb-b178-09ef0801fa10 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:39.510441 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b0df1a-adbc-42cb-b178-09ef0801fa10 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:39.519296 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b0df1a-adbc-42cb-b178-09ef0801fa10 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:39.520828 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b0df1a-adbc-42cb-b178-09ef0801fa10 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:39.536768 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-50b0df1a-adbc-42cb-b178-09ef0801fa10 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:23:39.536768 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:39.536768 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:39.536768 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:39.536768 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:39.536768 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:39.536768 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:39.536768 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:39.536768 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:39.536768 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:39.536768 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:39.536768 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:39.536768 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 15:23:39.536768 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 15:23:39.536768 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 15:23:39.538469 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 15:23:39.538469 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:39.538469 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:39.538469 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:39.538469 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:39.538469 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:39.538469 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:39.538469 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:39.538469 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:39.538469 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:39.538469 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:39.538469 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:39.538469 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:39.538469 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:39.538469 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:39.538469 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:39.538469 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:39.541446 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 15:23:39.541446 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 15:23:39.541446 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 15:23:39.541446 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 15:23:39.541446 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 15:23:39.541446 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 15:23:39.541446 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:23:39.541446 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:23:39.541446 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 438/863] 10.222.0.99 () {56 vars in 1111 bytes} [Thu Jul 14 15:23:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:39.545450 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71db336f-2887-482f-a4a1-5adac04ead60 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:39.651300 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71db336f-2887-482f-a4a1-5adac04ead60 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:39.689869 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e91b62d-7a25-4fa9-b86c-736b2ccc8501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:39.690546 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e91b62d-7a25-4fa9-b86c-736b2ccc8501 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:39.690818 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e91b62d-7a25-4fa9-b86c-736b2ccc8501 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:39.691106 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e91b62d-7a25-4fa9-b86c-736b2ccc8501 None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:39.691932 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e91b62d-7a25-4fa9-b86c-736b2ccc8501 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:39.692247 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e91b62d-7a25-4fa9-b86c-736b2ccc8501 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:39.693687 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e91b62d-7a25-4fa9-b86c-736b2ccc8501 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:39.703968 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:39.711707 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 426/864] 10.222.0.99 () {56 vars in 1148 bytes} [Thu Jul 14 15:23:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:39.714378 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b0df1a-adbc-42cb-b178-09ef0801fa10 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:39.748692 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:39.824066 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b0df1a-adbc-42cb-b178-09ef0801fa10 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:39.859830 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc9c430-e589-4a20-9e0b-192427ed82ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:39.860501 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc9c430-e589-4a20-9e0b-192427ed82ce None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:39.860793 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc9c430-e589-4a20-9e0b-192427ed82ce None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:39.861077 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc9c430-e589-4a20-9e0b-192427ed82ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:39.861927 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc9c430-e589-4a20-9e0b-192427ed82ce None admin] RBAC: Adding query filter params (role.id=d0f6ba1314f14d9f9c816bdb5d0c3aac, scope.project.id=9048d1a6f00443ec92f11471babe5a56, user.id=ddd2ec3b3ebc4aacb322c95270374451) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:39.862241 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc9c430-e589-4a20-9e0b-192427ed82ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:39.863717 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc9c430-e589-4a20-9e0b-192427ed82ce None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:39.875751 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 439/865] 10.222.0.99 () {56 vars in 1374 bytes} [Thu Jul 14 15:23:39 2022] GET /identity/v3/role_assignments?user.id=ddd2ec3b3ebc4aacb322c95270374451&scope.project.id=9048d1a6f00443ec92f11471babe5a56&role.id=d0f6ba1314f14d9f9c816bdb5d0c3aac => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:23:40.234240 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28269bc-f03d-4f03-9ec9-8ed4d13177b4 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:40.234878 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28269bc-f03d-4f03-9ec9-8ed4d13177b4 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:40.235175 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28269bc-f03d-4f03-9ec9-8ed4d13177b4 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:40.237940 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 427/866] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:23:40 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:23:40.246239 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1ec38d-116a-4553-8962-49d8a50f102d None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:40.246753 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1ec38d-116a-4553-8962-49d8a50f102d None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:40.247255 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1ec38d-116a-4553-8962-49d8a50f102d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:40.275772 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-dc1ec38d-116a-4553-8962-49d8a50f102d None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:23:40.277798 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-dc1ec38d-116a-4553-8962-49d8a50f102d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:23:40.380780 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 440/867] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:23:40 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 15:23:40.419721 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28269bc-f03d-4f03-9ec9-8ed4d13177b4 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:40.551423 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28269bc-f03d-4f03-9ec9-8ed4d13177b4 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:40.562239 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05816faa-fdd0-4914-82f4-6a302a08f2c8 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:40.563034 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05816faa-fdd0-4914-82f4-6a302a08f2c8 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:40.563529 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05816faa-fdd0-4914-82f4-6a302a08f2c8 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:40.565439 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 441/868] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:23:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:23:40.571960 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6294ebce-ab2a-4f5f-a4cd-7a293edc8aa4 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:40.572542 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6294ebce-ab2a-4f5f-a4cd-7a293edc8aa4 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:40.573053 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6294ebce-ab2a-4f5f-a4cd-7a293edc8aa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:40.590642 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6229c420-7bf2-43fe-a12c-0ce7ebd772d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:40.591463 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6229c420-7bf2-43fe-a12c-0ce7ebd772d7 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:40.591785 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6229c420-7bf2-43fe-a12c-0ce7ebd772d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:40.592072 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6229c420-7bf2-43fe-a12c-0ce7ebd772d7 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:40.593024 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6229c420-7bf2-43fe-a12c-0ce7ebd772d7 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:40.595086 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6229c420-7bf2-43fe-a12c-0ce7ebd772d7 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:40.595390 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-6294ebce-ab2a-4f5f-a4cd-7a293edc8aa4 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:23:40.596853 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-6294ebce-ab2a-4f5f-a4cd-7a293edc8aa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:23:40.600299 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:40.611515 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-6229c420-7bf2-43fe-a12c-0ce7ebd772d7 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 14 15:23:40.611515 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:40.611515 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 15:23:40.611515 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 15:23:40.611515 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 15:23:40.611515 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 15:23:40.611515 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 15:23:40.611515 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 15:23:40.611515 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 14 15:23:40.611515 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:23:40.611515 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 15:23:40.611515 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:23:40.611515 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:40.611515 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:40.611515 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:40.611515 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:40.612787 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:40.612787 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:40.612787 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:40.612787 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:40.612787 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:40.612787 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:40.612787 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:40.612787 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 15:23:40.612787 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 15:23:40.612787 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 15:23:40.612787 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 15:23:40.612787 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:40.612787 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:40.612787 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:40.612787 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:40.612787 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:40.613665 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:40.613665 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:40.613665 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:40.613665 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:40.613665 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:40.613665 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:40.613665 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:40.613665 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:40.613665 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:40.613665 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:40.613665 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:40.613665 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 15:23:40.613665 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 15:23:40.613665 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 14 15:23:40.613665 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:23:40.613665 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 428/869] 10.222.0.99 () {56 vars in 1132 bytes} [Thu Jul 14 15:23:40 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:40.621850 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6229c420-7bf2-43fe-a12c-0ce7ebd772d7 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:40.682270 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 442/870] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:23:40 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 15:23:40.705722 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6294ebce-ab2a-4f5f-a4cd-7a293edc8aa4 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:40.739951 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:40.812869 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6229c420-7bf2-43fe-a12c-0ce7ebd772d7 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:40.817753 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6294ebce-ab2a-4f5f-a4cd-7a293edc8aa4 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:40.877511 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5a9a2c-6e16-47fe-b98d-38ce1564f542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:40.878212 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5a9a2c-6e16-47fe-b98d-38ce1564f542 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:40.878503 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5a9a2c-6e16-47fe-b98d-38ce1564f542 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:40.878787 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5a9a2c-6e16-47fe-b98d-38ce1564f542 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:40.879524 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5a9a2c-6e16-47fe-b98d-38ce1564f542 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:40.880195 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5a9a2c-6e16-47fe-b98d-38ce1564f542 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:40.882646 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5a9a2c-6e16-47fe-b98d-38ce1564f542 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:40.889392 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 429/871] 10.222.0.99 () {56 vars in 1126 bytes} [Thu Jul 14 15:23:40 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:23:40.900966 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5a9a2c-6e16-47fe-b98d-38ce1564f542 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:41.010828 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25a1d6d-c1d1-44e0-8033-209051f5809c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:41.011821 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25a1d6d-c1d1-44e0-8033-209051f5809c None admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:41.012254 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25a1d6d-c1d1-44e0-8033-209051f5809c None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:41.012692 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25a1d6d-c1d1-44e0-8033-209051f5809c None admin] PATH_INFO: `/v3/services` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:41.013591 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25a1d6d-c1d1-44e0-8033-209051f5809c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:41.015159 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25a1d6d-c1d1-44e0-8033-209051f5809c None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:41.027711 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:41.035160 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 443/872] 10.222.0.99 () {60 vars in 1141 bytes} [Thu Jul 14 15:23:40 2022] POST /identity/v3/services => generated 251 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 15:23:41.052370 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5a9a2c-6e16-47fe-b98d-38ce1564f542 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:41.092797 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:41.101031 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52ecb0c-58df-4812-a53e-d69e8bb540ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:41.101720 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52ecb0c-58df-4812-a53e-d69e8bb540ce None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:41.102013 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52ecb0c-58df-4812-a53e-d69e8bb540ce None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:41.102297 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52ecb0c-58df-4812-a53e-d69e8bb540ce None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:41.103031 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52ecb0c-58df-4812-a53e-d69e8bb540ce None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:41.103343 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52ecb0c-58df-4812-a53e-d69e8bb540ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:41.104793 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52ecb0c-58df-4812-a53e-d69e8bb540ce None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:41.110922 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 430/873] 10.222.0.99 () {56 vars in 1126 bytes} [Thu Jul 14 15:23:40 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:41.436891 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7352e2-ad68-4a27-94c5-795d96b8977f None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:41.437571 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7352e2-ad68-4a27-94c5-795d96b8977f None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:41.437571 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7352e2-ad68-4a27-94c5-795d96b8977f None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:41.439059 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 444/874] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:23:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:23:41.444412 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0e3859-c11a-4cb4-80f4-9d3dc9597486 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:41.445036 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0e3859-c11a-4cb4-80f4-9d3dc9597486 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:41.445036 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0e3859-c11a-4cb4-80f4-9d3dc9597486 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:41.462817 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-1d0e3859-c11a-4cb4-80f4-9d3dc9597486 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:23:41.464030 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-1d0e3859-c11a-4cb4-80f4-9d3dc9597486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:23:41.534527 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 431/875] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:23:41 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 15:23:41.557993 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7352e2-ad68-4a27-94c5-795d96b8977f None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:41.670775 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7352e2-ad68-4a27-94c5-795d96b8977f None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:41.715945 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8401c2cf-8a97-48c4-adff-15a893868880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:41.716706 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8401c2cf-8a97-48c4-adff-15a893868880 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:41.716994 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8401c2cf-8a97-48c4-adff-15a893868880 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:41.717274 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8401c2cf-8a97-48c4-adff-15a893868880 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:41.729994 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8401c2cf-8a97-48c4-adff-15a893868880 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:23:41.740723 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-8401c2cf-8a97-48c4-adff-15a893868880 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 15:23:41.740723 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:41.740723 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:41.740723 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:41.740723 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:41.740723 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:41.740723 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:41.740723 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:41.740723 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:41.740723 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:41.740723 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:41.740723 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:41.740723 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:23:41.740723 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:23:41.740723 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:23:41.740723 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:23:41.741734 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:23:41.741734 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:23:41.741734 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:23:41.741734 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:23:41.741734 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:41.741734 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:41.741734 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:41.741734 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:41.741734 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:41.741734 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:41.741734 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:41.741734 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:41.741734 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:41.741734 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:41.741734 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:41.741734 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:41.742765 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:41.742765 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:41.742765 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:41.742765 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:41.742765 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:23:41.742765 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:23:41.742765 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:23:41.742765 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:23:41.742765 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:23:41.742765 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:23:41.742765 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 15:23:41.742765 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:23:41.743477 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 445/876] 10.222.0.99 () {56 vars in 1120 bytes} [Thu Jul 14 15:23:41 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:41.752187 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0e3859-c11a-4cb4-80f4-9d3dc9597486 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:41.856846 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0e3859-c11a-4cb4-80f4-9d3dc9597486 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:41.879985 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:41.895149 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53375b1-ba5a-4d49-ae5c-41a099f1b50f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:41.895858 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53375b1-ba5a-4d49-ae5c-41a099f1b50f None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:41.896150 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53375b1-ba5a-4d49-ae5c-41a099f1b50f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:41.896430 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53375b1-ba5a-4d49-ae5c-41a099f1b50f None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:41.897188 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53375b1-ba5a-4d49-ae5c-41a099f1b50f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:41.897498 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53375b1-ba5a-4d49-ae5c-41a099f1b50f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:41.898877 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53375b1-ba5a-4d49-ae5c-41a099f1b50f None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:41.907972 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 432/877] 10.222.0.99 () {56 vars in 1115 bytes} [Thu Jul 14 15:23:41 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:41.917019 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8401c2cf-8a97-48c4-adff-15a893868880 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:41.984702 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:42.028310 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8401c2cf-8a97-48c4-adff-15a893868880 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:42.063887 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab729813-28b1-4e6b-8d63-22f80d6c2739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:42.064696 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab729813-28b1-4e6b-8d63-22f80d6c2739 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:42.064985 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab729813-28b1-4e6b-8d63-22f80d6c2739 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:42.065295 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab729813-28b1-4e6b-8d63-22f80d6c2739 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:42.076635 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab729813-28b1-4e6b-8d63-22f80d6c2739 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:42.078632 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab729813-28b1-4e6b-8d63-22f80d6c2739 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:42.097085 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-ab729813-28b1-4e6b-8d63-22f80d6c2739 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:23:42.097085 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:42.097085 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:42.097085 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:42.097085 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:42.097085 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:42.097085 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:42.097085 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:42.097085 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:42.097085 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:42.097085 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:42.097085 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:42.097085 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:23:42.097085 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:23:42.097085 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:23:42.097998 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:23:42.097998 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:42.097998 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:42.097998 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:42.097998 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:42.097998 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:42.097998 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:42.097998 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:42.097998 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:42.097998 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:42.097998 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:42.097998 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:42.097998 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:42.097998 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:42.097998 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:42.097998 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:42.097998 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:42.098885 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:23:42.098885 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:23:42.098885 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:23:42.098885 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:23:42.098885 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 446/878] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:23:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:42.104273 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53375b1-ba5a-4d49-ae5c-41a099f1b50f None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:42.209175 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53375b1-ba5a-4d49-ae5c-41a099f1b50f None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:42.212160 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9d71c6-0beb-418d-8a86-69d7de4b00fc None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:42.212461 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9d71c6-0beb-418d-8a86-69d7de4b00fc None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:42.212746 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9d71c6-0beb-418d-8a86-69d7de4b00fc None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:42.214352 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 447/879] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:23:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 15:23:42.221758 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68069010-d7d3-4a1e-88eb-fa01be26c706 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:42.222165 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68069010-d7d3-4a1e-88eb-fa01be26c706 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:42.222524 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68069010-d7d3-4a1e-88eb-fa01be26c706 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:42.240610 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-68069010-d7d3-4a1e-88eb-fa01be26c706 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:23:42.242416 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-68069010-d7d3-4a1e-88eb-fa01be26c706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:23:42.248162 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef76f06-3d23-426b-8fdd-5f2ee2ba212b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:42.249107 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef76f06-3d23-426b-8fdd-5f2ee2ba212b None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:42.249832 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef76f06-3d23-426b-8fdd-5f2ee2ba212b None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:42.250435 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef76f06-3d23-426b-8fdd-5f2ee2ba212b None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:42.251414 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef76f06-3d23-426b-8fdd-5f2ee2ba212b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:42.251928 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef76f06-3d23-426b-8fdd-5f2ee2ba212b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:42.253501 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef76f06-3d23-426b-8fdd-5f2ee2ba212b None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:42.268505 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 433/880] 10.222.0.99 () {56 vars in 1109 bytes} [Thu Jul 14 15:23:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:42.275881 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef76f06-3d23-426b-8fdd-5f2ee2ba212b None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:42.331247 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:42.336654 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 448/881] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:23:42 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 15:23:42.358092 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68069010-d7d3-4a1e-88eb-fa01be26c706 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:42.414163 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:42.432659 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef76f06-3d23-426b-8fdd-5f2ee2ba212b None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:42.479035 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68069010-d7d3-4a1e-88eb-fa01be26c706 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:42.485839 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a50efbc-ba72-4deb-aa73-e07015a08f3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:42.487232 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a50efbc-ba72-4deb-aa73-e07015a08f3e None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:42.488123 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a50efbc-ba72-4deb-aa73-e07015a08f3e None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:42.488557 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a50efbc-ba72-4deb-aa73-e07015a08f3e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:42.502110 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a50efbc-ba72-4deb-aa73-e07015a08f3e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:42.504676 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a50efbc-ba72-4deb-aa73-e07015a08f3e None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:42.517991 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d31fe3b-3801-4964-848a-5a238c1a58b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:42.519179 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d31fe3b-3801-4964-848a-5a238c1a58b3 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:42.519755 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d31fe3b-3801-4964-848a-5a238c1a58b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:42.520269 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d31fe3b-3801-4964-848a-5a238c1a58b3 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:42.521127 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d31fe3b-3801-4964-848a-5a238c1a58b3 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:42.522743 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d31fe3b-3801-4964-848a-5a238c1a58b3 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:42.525173 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-1a50efbc-ba72-4deb-aa73-e07015a08f3e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 15:23:42.525173 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:42.525173 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:42.525173 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:42.525173 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:42.525173 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:42.525173 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:42.525173 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:42.525173 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:42.525173 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:42.525173 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:42.525173 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:42.525173 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 15:23:42.525173 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 15:23:42.525173 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 15:23:42.525173 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 15:23:42.526137 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:42.526137 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:42.526137 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 15:23:42.526137 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:23:42.526137 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 15:23:42.526137 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:23:42.526137 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:42.526137 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:42.526137 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:42.526137 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:42.526137 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:42.526137 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:42.526137 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:42.526137 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:42.526137 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:42.526137 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:42.526137 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:42.526137 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:42.527075 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:42.527075 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:42.527075 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 15:23:42.527075 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 15:23:42.527075 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 15:23:42.527075 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 15:23:42.527075 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 15:23:42.527075 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 15:23:42.527075 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 15:23:42.527075 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:23:42.531675 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 434/882] 10.222.0.99 () {56 vars in 1117 bytes} [Thu Jul 14 15:23:42 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:42.534162 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a50efbc-ba72-4deb-aa73-e07015a08f3e None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:42.543237 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-8d31fe3b-3801-4964-848a-5a238c1a58b3 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 15:23:42.543237 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:42.543237 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 15:23:42.543237 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 15:23:42.543237 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 15:23:42.543237 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 15:23:42.543237 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 15:23:42.543237 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 15:23:42.543237 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 15:23:42.543237 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:23:42.543237 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 15:23:42.543237 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:23:42.543237 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:42.543237 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:42.543237 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:42.543237 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:42.543237 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:42.544254 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:42.544254 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:42.544254 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:42.544254 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:42.544254 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:42.544254 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:42.544254 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 15:23:42.544254 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 15:23:42.544254 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 15:23:42.544254 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 15:23:42.544254 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:42.544254 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:42.544254 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:42.544254 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:42.544254 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:42.544254 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:42.545062 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:42.545062 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:42.545062 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:42.545062 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:42.545062 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:42.545062 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:42.545062 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:42.545062 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:42.545062 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:42.545062 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:42.545062 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 15:23:42.545062 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 15:23:42.545062 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 15:23:42.545062 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:23:42.553196 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 449/883] 10.222.0.99 () {56 vars in 1126 bytes} [Thu Jul 14 15:23:42 2022] GET /identity/v3/services/object-store => generated 93 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 15:23:42.554304 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d31fe3b-3801-4964-848a-5a238c1a58b3 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:42.665563 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d31fe3b-3801-4964-848a-5a238c1a58b3 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:42.669206 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:42.674181 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a50efbc-ba72-4deb-aa73-e07015a08f3e None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:42.676214 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:42.711556 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cf6b34-05f0-4c7b-8463-e842c35d5685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:42.712621 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cf6b34-05f0-4c7b-8463-e842c35d5685 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:42.713110 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cf6b34-05f0-4c7b-8463-e842c35d5685 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:42.713584 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cf6b34-05f0-4c7b-8463-e842c35d5685 None admin] PATH_INFO: `/v3/services` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:42.714538 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cf6b34-05f0-4c7b-8463-e842c35d5685 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:42.715042 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cf6b34-05f0-4c7b-8463-e842c35d5685 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:42.716658 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cf6b34-05f0-4c7b-8463-e842c35d5685 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:42.726990 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13083c5-34f8-4a6a-89ea-58f974c3f758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:42.726990 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13083c5-34f8-4a6a-89ea-58f974c3f758 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:42.726990 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13083c5-34f8-4a6a-89ea-58f974c3f758 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:42.726990 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13083c5-34f8-4a6a-89ea-58f974c3f758 None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:42.727427 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13083c5-34f8-4a6a-89ea-58f974c3f758 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:42.727427 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13083c5-34f8-4a6a-89ea-58f974c3f758 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:42.728029 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 450/884] 10.222.0.99 () {56 vars in 1122 bytes} [Thu Jul 14 15:23:42 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:42.728598 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13083c5-34f8-4a6a-89ea-58f974c3f758 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:42.729382 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cf6b34-05f0-4c7b-8463-e842c35d5685 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:42.755966 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 435/885] 10.222.0.99 () {56 vars in 1149 bytes} [Thu Jul 14 15:23:42 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:23:42.759244 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13083c5-34f8-4a6a-89ea-58f974c3f758 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:42.879662 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cf6b34-05f0-4c7b-8463-e842c35d5685 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:42.910538 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13083c5-34f8-4a6a-89ea-58f974c3f758 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:42.917535 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca3ea39-2bfb-4de2-b74e-afeff4f4eaec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:42.918327 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca3ea39-2bfb-4de2-b74e-afeff4f4eaec None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:42.918617 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca3ea39-2bfb-4de2-b74e-afeff4f4eaec None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:42.918900 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca3ea39-2bfb-4de2-b74e-afeff4f4eaec None admin] PATH_INFO: `/v3/services` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:42.919714 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca3ea39-2bfb-4de2-b74e-afeff4f4eaec None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:42.920029 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca3ea39-2bfb-4de2-b74e-afeff4f4eaec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:42.922921 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca3ea39-2bfb-4de2-b74e-afeff4f4eaec None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:42.933720 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 451/886] 10.222.0.99 () {56 vars in 1122 bytes} [Thu Jul 14 15:23:42 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:42.957896 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563e5b4e-ae4d-4b30-8670-093a9b5ae11a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:42.958993 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563e5b4e-ae4d-4b30-8670-093a9b5ae11a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:42.959494 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563e5b4e-ae4d-4b30-8670-093a9b5ae11a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:42.960009 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563e5b4e-ae4d-4b30-8670-093a9b5ae11a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:42.995190 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:43.046217 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563e5b4e-ae4d-4b30-8670-093a9b5ae11a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:43.048355 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563e5b4e-ae4d-4b30-8670-093a9b5ae11a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:43.164257 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 436/887] 10.222.0.99 () {58 vars in 1282 bytes} [Thu Jul 14 15:23:42 2022] GET /identity/v3/auth/tokens => generated 1252 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 14 15:23:43.169906 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca3ea39-2bfb-4de2-b74e-afeff4f4eaec None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:43.211221 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:43.288094 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca3ea39-2bfb-4de2-b74e-afeff4f4eaec None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:43.323574 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a18c46-a085-4356-ac04-898a253b531b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:43.324326 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a18c46-a085-4356-ac04-898a253b531b None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:43.324611 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a18c46-a085-4356-ac04-898a253b531b None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:43.324887 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a18c46-a085-4356-ac04-898a253b531b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:43.336418 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a18c46-a085-4356-ac04-898a253b531b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:43.337881 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a18c46-a085-4356-ac04-898a253b531b None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:43.356924 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-d6a18c46-a085-4356-ac04-898a253b531b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:23:43.356924 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:43.356924 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:43.356924 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:43.356924 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:43.356924 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:43.356924 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:43.356924 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:43.356924 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:43.356924 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:43.356924 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:43.356924 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:43.356924 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:23:43.356924 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:23:43.356924 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:23:43.358057 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:23:43.358057 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:43.358057 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:43.358057 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:43.358057 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:43.358057 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:43.358057 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:43.358057 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:43.358057 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:43.358057 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:43.358057 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:43.358057 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:43.358057 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:43.358057 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:43.358057 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:43.358057 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:43.358057 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:43.359896 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:23:43.359896 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:23:43.359896 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:23:43.359896 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:23:43.359896 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 452/888] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:23:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:43.364768 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563e5b4e-ae4d-4b30-8670-093a9b5ae11a None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:43.396364 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7d56e2-b8e3-4dd6-9522-0197492dbcef None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:43.396692 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7d56e2-b8e3-4dd6-9522-0197492dbcef None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:43.396917 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7d56e2-b8e3-4dd6-9522-0197492dbcef None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:43.398680 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 453/889] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:23:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:23:43.401443 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b711ba-cb36-4746-a615-eedf4b2fa3ec None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:43.401992 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b711ba-cb36-4746-a615-eedf4b2fa3ec None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:43.402481 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b711ba-cb36-4746-a615-eedf4b2fa3ec None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:43.403942 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 454/890] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:23:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 15:23:43.405015 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f64aa3-ad60-4ef2-bf35-aeb80dedac25 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:43.405306 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f64aa3-ad60-4ef2-bf35-aeb80dedac25 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:43.406096 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f64aa3-ad60-4ef2-bf35-aeb80dedac25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:43.424492 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-a4f64aa3-ad60-4ef2-bf35-aeb80dedac25 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:23:43.425694 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-a4f64aa3-ad60-4ef2-bf35-aeb80dedac25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:23:43.431830 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:43.479188 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563e5b4e-ae4d-4b30-8670-093a9b5ae11a None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:43.506399 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 455/891] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:23:43 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 15:23:43.509464 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93ee1ef-b67e-4c4d-9e66-f9772b51e64a None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:43.510044 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93ee1ef-b67e-4c4d-9e66-f9772b51e64a None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:43.510680 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93ee1ef-b67e-4c4d-9e66-f9772b51e64a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:43.521547 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1afbaf-97aa-4ad0-8883-8276f68f6518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:43.522898 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1afbaf-97aa-4ad0-8883-8276f68f6518 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:43.523440 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1afbaf-97aa-4ad0-8883-8276f68f6518 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:43.523949 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1afbaf-97aa-4ad0-8883-8276f68f6518 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:43.524930 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1afbaf-97aa-4ad0-8883-8276f68f6518 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:43.525438 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1afbaf-97aa-4ad0-8883-8276f68f6518 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:43.527070 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1afbaf-97aa-4ad0-8883-8276f68f6518 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:43.536542 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-d93ee1ef-b67e-4c4d-9e66-f9772b51e64a None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:23:43.537757 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-d93ee1ef-b67e-4c4d-9e66-f9772b51e64a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:23:43.540576 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 437/892] 10.222.0.99 () {56 vars in 1109 bytes} [Thu Jul 14 15:23:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:43.541282 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1afbaf-97aa-4ad0-8883-8276f68f6518 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:43.550015 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:43.612993 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 456/893] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:23:43 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 15:23:43.613789 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93ee1ef-b67e-4c4d-9e66-f9772b51e64a None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:43.658129 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1afbaf-97aa-4ad0-8883-8276f68f6518 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:43.678585 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:43.699000 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8189afff-80f6-45d7-affd-011a7ff260e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:43.699854 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8189afff-80f6-45d7-affd-011a7ff260e8 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:43.700146 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8189afff-80f6-45d7-affd-011a7ff260e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:43.700426 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8189afff-80f6-45d7-affd-011a7ff260e8 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:43.701095 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8189afff-80f6-45d7-affd-011a7ff260e8 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:43.703090 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8189afff-80f6-45d7-affd-011a7ff260e8 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:43.716634 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-8189afff-80f6-45d7-affd-011a7ff260e8 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 14 15:23:43.716634 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:43.716634 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 15:23:43.716634 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 15:23:43.716634 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 15:23:43.716634 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 15:23:43.716634 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 15:23:43.716634 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 15:23:43.716634 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 14 15:23:43.716634 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:23:43.716634 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 15:23:43.716634 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:23:43.716634 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:43.716634 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:43.716634 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:43.716634 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:43.716634 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:43.717872 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:43.717872 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:43.717872 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:43.717872 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:43.717872 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:43.717872 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:43.717872 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 15:23:43.717872 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 15:23:43.717872 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 15:23:43.717872 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 15:23:43.717872 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:43.717872 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:43.717872 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:43.717872 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:43.717872 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:43.717872 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:43.720085 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:43.720085 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:43.720085 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:43.720085 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:43.720085 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:43.720085 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:43.720085 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:43.720085 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:43.720085 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:43.720085 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:43.720085 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 15:23:43.720085 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 15:23:43.720085 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 14 15:23:43.720085 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:23:43.720085 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 438/894] 10.222.0.99 () {56 vars in 1114 bytes} [Thu Jul 14 15:23:43 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:43.721043 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8189afff-80f6-45d7-affd-011a7ff260e8 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:43.753062 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93ee1ef-b67e-4c4d-9e66-f9772b51e64a None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:43.792238 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3512eaab-81b1-4c31-a4ae-efcfb498c4ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:43.793315 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3512eaab-81b1-4c31-a4ae-efcfb498c4ac None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:43.793751 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3512eaab-81b1-4c31-a4ae-efcfb498c4ac None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:43.794189 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3512eaab-81b1-4c31-a4ae-efcfb498c4ac None admin] PATH_INFO: `/v3/projects/service` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:43.805888 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3512eaab-81b1-4c31-a4ae-efcfb498c4ac None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:43.808141 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3512eaab-81b1-4c31-a4ae-efcfb498c4ac None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:43.823879 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-3512eaab-81b1-4c31-a4ae-efcfb498c4ac None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:23:43.823879 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:43.823879 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:43.823879 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:43.823879 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:43.823879 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:43.823879 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:43.823879 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:43.823879 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:43.823879 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:43.823879 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:43.823879 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:43.823879 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 15:23:43.823879 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 15:23:43.823879 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 15:23:43.825519 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 15:23:43.825519 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:43.825519 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:43.825519 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:43.825519 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:43.825519 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:43.825519 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:43.825519 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:43.825519 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:43.825519 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:43.825519 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:43.825519 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:43.825519 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:43.825519 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:43.825519 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:43.825519 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:43.825519 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:43.827405 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 15:23:43.827405 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 15:23:43.827405 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 15:23:43.827405 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 15:23:43.827405 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 15:23:43.827405 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 15:23:43.827405 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:23:43.827405 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:23:43.827405 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 457/895] 10.222.0.99 () {56 vars in 1111 bytes} [Thu Jul 14 15:23:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:43.827405 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3512eaab-81b1-4c31-a4ae-efcfb498c4ac None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:43.837262 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8189afff-80f6-45d7-affd-011a7ff260e8 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:43.841434 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:43.876271 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a4b66e-1bdc-4a3b-ae7d-e115fac3440e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:43.876857 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a4b66e-1bdc-4a3b-ae7d-e115fac3440e None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:43.877143 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a4b66e-1bdc-4a3b-ae7d-e115fac3440e None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:43.877423 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a4b66e-1bdc-4a3b-ae7d-e115fac3440e None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:43.878146 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a4b66e-1bdc-4a3b-ae7d-e115fac3440e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:43.879524 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a4b66e-1bdc-4a3b-ae7d-e115fac3440e None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:43.897054 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 439/896] 10.222.0.99 () {60 vars in 1141 bytes} [Thu Jul 14 15:23:43 2022] POST /identity/v3/services => generated 269 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:23:43.908845 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a4b66e-1bdc-4a3b-ae7d-e115fac3440e None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:43.960991 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3512eaab-81b1-4c31-a4ae-efcfb498c4ac None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:43.986380 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:44.021728 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd49abb1-ccc0-491b-bbe9-0947a3731599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:44.022527 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd49abb1-ccc0-491b-bbe9-0947a3731599 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:44.022837 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd49abb1-ccc0-491b-bbe9-0947a3731599 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:44.023068 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd49abb1-ccc0-491b-bbe9-0947a3731599 None admin] PATH_INFO: `/v3/services` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:44.023864 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd49abb1-ccc0-491b-bbe9-0947a3731599 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:44.024179 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd49abb1-ccc0-491b-bbe9-0947a3731599 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:44.025611 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd49abb1-ccc0-491b-bbe9-0947a3731599 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:44.036417 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 458/897] 10.222.0.99 () {56 vars in 1114 bytes} [Thu Jul 14 15:23:43 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:44.038959 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd49abb1-ccc0-491b-bbe9-0947a3731599 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:44.078341 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a4b66e-1bdc-4a3b-ae7d-e115fac3440e None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:44.119215 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:44.120183 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dd08e8-eb40-4eb1-a80d-213bbce32f41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:44.120448 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dd08e8-eb40-4eb1-a80d-213bbce32f41 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:44.120743 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dd08e8-eb40-4eb1-a80d-213bbce32f41 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:44.121027 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dd08e8-eb40-4eb1-a80d-213bbce32f41 None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:44.121870 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dd08e8-eb40-4eb1-a80d-213bbce32f41 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:44.122188 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dd08e8-eb40-4eb1-a80d-213bbce32f41 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:44.125563 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dd08e8-eb40-4eb1-a80d-213bbce32f41 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:44.142408 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 440/898] 10.222.0.99 () {56 vars in 1148 bytes} [Thu Jul 14 15:23:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:44.148606 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dd08e8-eb40-4eb1-a80d-213bbce32f41 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:44.176532 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd49abb1-ccc0-491b-bbe9-0947a3731599 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:44.226303 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df8c6f6-97d8-4cb4-9fc3-091911db9075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:44.227870 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df8c6f6-97d8-4cb4-9fc3-091911db9075 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:44.228684 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df8c6f6-97d8-4cb4-9fc3-091911db9075 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:44.229189 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df8c6f6-97d8-4cb4-9fc3-091911db9075 None admin] PATH_INFO: `/v3/services` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:44.230362 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df8c6f6-97d8-4cb4-9fc3-091911db9075 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:44.230859 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df8c6f6-97d8-4cb4-9fc3-091911db9075 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:44.233423 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df8c6f6-97d8-4cb4-9fc3-091911db9075 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:44.245670 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 459/899] 10.222.0.99 () {56 vars in 1114 bytes} [Thu Jul 14 15:23:44 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:44.252631 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df8c6f6-97d8-4cb4-9fc3-091911db9075 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:44.281503 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dd08e8-eb40-4eb1-a80d-213bbce32f41 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:44.283152 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:44.321848 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0f3615-af00-412e-b08e-b04217962e12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:44.322513 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0f3615-af00-412e-b08e-b04217962e12 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:44.322801 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0f3615-af00-412e-b08e-b04217962e12 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:44.323069 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0f3615-af00-412e-b08e-b04217962e12 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:44.323936 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0f3615-af00-412e-b08e-b04217962e12 None admin] RBAC: Adding query filter params (role.id=ebca1dcd25d64cebaae5f4c18285be4a, scope.project.id=9048d1a6f00443ec92f11471babe5a56, user.id=adaeadf613954d7393b3fd3bae68c362) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:44.324455 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0f3615-af00-412e-b08e-b04217962e12 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:44.325666 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0f3615-af00-412e-b08e-b04217962e12 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:44.331894 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 441/900] 10.222.0.99 () {56 vars in 1374 bytes} [Thu Jul 14 15:23:44 2022] GET /identity/v3/role_assignments?user.id=adaeadf613954d7393b3fd3bae68c362&scope.project.id=9048d1a6f00443ec92f11471babe5a56&role.id=ebca1dcd25d64cebaae5f4c18285be4a => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:44.376609 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df8c6f6-97d8-4cb4-9fc3-091911db9075 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:44.412850 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:44.416208 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2839fe3-eb3c-4380-a76e-abd58bf32e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:44.417159 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2839fe3-eb3c-4380-a76e-abd58bf32e18 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:44.417687 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2839fe3-eb3c-4380-a76e-abd58bf32e18 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:44.418178 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2839fe3-eb3c-4380-a76e-abd58bf32e18 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:44.419206 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2839fe3-eb3c-4380-a76e-abd58bf32e18 None admin] RBAC: Adding query filter params (interface=public, service_id=55cbeffb8b534e96946615793b6b7caf, region_id=RegionOne) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:44.419814 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2839fe3-eb3c-4380-a76e-abd58bf32e18 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:44.421973 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2839fe3-eb3c-4380-a76e-abd58bf32e18 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:44.429085 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 460/901] 10.222.0.99 () {56 vars in 1251 bytes} [Thu Jul 14 15:23:44 2022] GET /identity/v3/endpoints?service_id=55cbeffb8b534e96946615793b6b7caf&interface=public®ion_id=RegionOne => generated 194 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:23:44.435032 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0f3615-af00-412e-b08e-b04217962e12 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:44.553599 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0f3615-af00-412e-b08e-b04217962e12 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:44.594014 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37091179-d687-4af5-86dc-497cd4c2d57c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:44.595075 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37091179-d687-4af5-86dc-497cd4c2d57c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:44.595571 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37091179-d687-4af5-86dc-497cd4c2d57c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:44.596118 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37091179-d687-4af5-86dc-497cd4c2d57c None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:44.597019 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37091179-d687-4af5-86dc-497cd4c2d57c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:44.598594 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37091179-d687-4af5-86dc-497cd4c2d57c None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:44.605097 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 442/902] 10.222.0.99 () {56 vars in 1087 bytes} [Thu Jul 14 15:23:44 2022] GET /identity/v3/services => generated 1268 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:23:45.080734 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7110be-6e9d-4a4b-a993-584f43cb4636 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:45.081252 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7110be-6e9d-4a4b-a993-584f43cb4636 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:45.081370 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7110be-6e9d-4a4b-a993-584f43cb4636 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:45.082898 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 461/903] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:23:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:23:45.088601 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec99dadd-cc70-4f0e-bca2-0288c01c4c30 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:45.088892 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec99dadd-cc70-4f0e-bca2-0288c01c4c30 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:45.089178 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec99dadd-cc70-4f0e-bca2-0288c01c4c30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:45.106682 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-ec99dadd-cc70-4f0e-bca2-0288c01c4c30 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:23:45.107901 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ec99dadd-cc70-4f0e-bca2-0288c01c4c30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:23:45.134775 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:45.185027 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 443/904] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:23:45 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 15:23:45.208197 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7110be-6e9d-4a4b-a993-584f43cb4636 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:45.312630 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7110be-6e9d-4a4b-a993-584f43cb4636 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:45.347566 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bad2706-d0c8-45a6-a62e-f429380d95af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:45.348255 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bad2706-d0c8-45a6-a62e-f429380d95af None admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:45.348543 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bad2706-d0c8-45a6-a62e-f429380d95af None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:45.348822 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bad2706-d0c8-45a6-a62e-f429380d95af None admin] PATH_INFO: `/v3/services` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:45.349563 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bad2706-d0c8-45a6-a62e-f429380d95af None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:45.350920 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bad2706-d0c8-45a6-a62e-f429380d95af None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:45.371783 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 462/905] 10.222.0.99 () {60 vars in 1141 bytes} [Thu Jul 14 15:23:45 2022] POST /identity/v3/services => generated 241 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:23:46.146606 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5d753e-3743-4463-add0-e8c34535aa15 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:46.147124 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5d753e-3743-4463-add0-e8c34535aa15 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:46.147203 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5d753e-3743-4463-add0-e8c34535aa15 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:46.148845 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 444/906] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:23:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:23:46.154423 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156c625a-be7d-4ec5-a792-f6912485a1ce None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:46.154902 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156c625a-be7d-4ec5-a792-f6912485a1ce None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:46.154982 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156c625a-be7d-4ec5-a792-f6912485a1ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:46.184944 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-156c625a-be7d-4ec5-a792-f6912485a1ce None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:23:46.186155 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-156c625a-be7d-4ec5-a792-f6912485a1ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:23:46.240356 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:46.269083 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 463/907] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:23:46 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 15:23:46.293931 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5d753e-3743-4463-add0-e8c34535aa15 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:46.407204 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5d753e-3743-4463-add0-e8c34535aa15 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:46.444798 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b446dd1-04fc-4550-bb63-a19a5b5d5378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:46.445968 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b446dd1-04fc-4550-bb63-a19a5b5d5378 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:46.446465 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b446dd1-04fc-4550-bb63-a19a5b5d5378 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:46.446974 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b446dd1-04fc-4550-bb63-a19a5b5d5378 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:46.447870 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b446dd1-04fc-4550-bb63-a19a5b5d5378 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:46.449518 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b446dd1-04fc-4550-bb63-a19a5b5d5378 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:46.462647 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-1b446dd1-04fc-4550-bb63-a19a5b5d5378 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 14 15:23:46.462647 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:46.462647 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 15:23:46.462647 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 15:23:46.462647 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 15:23:46.462647 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 15:23:46.462647 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 15:23:46.462647 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 15:23:46.462647 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 14 15:23:46.462647 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:23:46.462647 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 15:23:46.462647 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:23:46.462647 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:46.462647 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:46.462647 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:46.462647 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:46.463696 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:46.463696 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:46.463696 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:46.463696 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:46.463696 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:46.463696 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:46.463696 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:46.463696 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 15:23:46.463696 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 15:23:46.463696 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 15:23:46.463696 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 15:23:46.463696 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:46.463696 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:46.463696 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:46.463696 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:46.463696 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:46.464535 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:46.464535 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:46.464535 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:46.464535 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:46.464535 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:46.464535 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:46.464535 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:46.464535 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:46.464535 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:46.464535 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:46.464535 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:46.464535 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 15:23:46.464535 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 15:23:46.464535 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 14 15:23:46.464535 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:23:46.467424 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 445/908] 10.222.0.99 () {56 vars in 1132 bytes} [Thu Jul 14 15:23:46 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:46.479668 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156c625a-be7d-4ec5-a792-f6912485a1ce None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:46.501142 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bf9ebb-662a-4959-81be-64f5a5770be9 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:46.501534 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bf9ebb-662a-4959-81be-64f5a5770be9 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:46.501706 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bf9ebb-662a-4959-81be-64f5a5770be9 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:46.503349 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 446/909] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:23:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 15:23:46.508620 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c311f086-6d17-4424-acc9-86bd55e93205 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:46.508914 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c311f086-6d17-4424-acc9-86bd55e93205 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:46.509201 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c311f086-6d17-4424-acc9-86bd55e93205 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:46.526903 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-c311f086-6d17-4424-acc9-86bd55e93205 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:23:46.528116 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c311f086-6d17-4424-acc9-86bd55e93205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:23:46.561035 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:46.610607 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 447/910] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:23:46 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 15:23:46.615784 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156c625a-be7d-4ec5-a792-f6912485a1ce None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:46.632609 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c311f086-6d17-4424-acc9-86bd55e93205 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:46.657969 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd4849b-9d63-468c-84db-9835af02a196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:46.659153 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd4849b-9d63-468c-84db-9835af02a196 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:46.659458 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd4849b-9d63-468c-84db-9835af02a196 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:46.659809 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd4849b-9d63-468c-84db-9835af02a196 None admin] PATH_INFO: `/v3/services` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:46.660571 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd4849b-9d63-468c-84db-9835af02a196 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:46.660885 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd4849b-9d63-468c-84db-9835af02a196 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:46.662336 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd4849b-9d63-468c-84db-9835af02a196 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:46.670193 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 464/911] 10.222.0.99 () {56 vars in 1126 bytes} [Thu Jul 14 15:23:46 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:46.676663 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd4849b-9d63-468c-84db-9835af02a196 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:46.738544 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:46.760926 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c311f086-6d17-4424-acc9-86bd55e93205 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:46.802819 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd4849b-9d63-468c-84db-9835af02a196 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:46.808979 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55fd932-03f7-497d-a9e3-499a5fc5fa4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:46.809798 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55fd932-03f7-497d-a9e3-499a5fc5fa4c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:46.810085 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55fd932-03f7-497d-a9e3-499a5fc5fa4c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:46.810358 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55fd932-03f7-497d-a9e3-499a5fc5fa4c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:46.824981 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55fd932-03f7-497d-a9e3-499a5fc5fa4c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:23:46.837142 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-d55fd932-03f7-497d-a9e3-499a5fc5fa4c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 15:23:46.837142 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:46.837142 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:46.837142 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:46.837142 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:46.837142 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:46.837142 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:46.837142 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:46.837142 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:46.837142 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:46.837142 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:46.837142 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:46.837142 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:23:46.837142 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:23:46.837142 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:23:46.837142 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:23:46.838142 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:23:46.838142 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:23:46.838142 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:23:46.838142 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:23:46.838142 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:46.838142 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:46.838142 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:46.838142 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:46.838142 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:46.838142 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:46.838142 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:46.838142 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:46.838142 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:46.838142 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:46.838142 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:46.838142 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:46.838953 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:46.838953 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:46.838953 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:46.838953 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:46.838953 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:23:46.838953 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:23:46.838953 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:23:46.838953 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:23:46.838953 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:23:46.838953 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:23:46.838953 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 15:23:46.838953 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:23:46.838953 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 448/912] 10.222.0.99 () {56 vars in 1120 bytes} [Thu Jul 14 15:23:46 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 15:23:46.844336 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfcdc3b-c202-44c5-b8aa-cac5b8f44788 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:46.844336 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfcdc3b-c202-44c5-b8aa-cac5b8f44788 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:46.844336 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfcdc3b-c202-44c5-b8aa-cac5b8f44788 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:46.844654 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a470950-0523-40da-a776-8cd76b791352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:46.844780 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 449/913] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:23:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:23:46.844854 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a470950-0523-40da-a776-8cd76b791352 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:46.844854 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a470950-0523-40da-a776-8cd76b791352 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:46.844854 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a470950-0523-40da-a776-8cd76b791352 None admin] PATH_INFO: `/v3/services` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:46.845228 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a470950-0523-40da-a776-8cd76b791352 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:46.846697 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a470950-0523-40da-a776-8cd76b791352 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:46.847474 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a470950-0523-40da-a776-8cd76b791352 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:46.847931 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfcdc3b-c202-44c5-b8aa-cac5b8f44788 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:46.855941 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 465/914] 10.222.0.99 () {56 vars in 1126 bytes} [Thu Jul 14 15:23:46 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:23:46.861182 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89b0abb-aa22-4dd3-b7ef-b0dead5202f3 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:46.861812 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89b0abb-aa22-4dd3-b7ef-b0dead5202f3 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:46.862102 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89b0abb-aa22-4dd3-b7ef-b0dead5202f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:46.869365 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:46.880070 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-b89b0abb-aa22-4dd3-b7ef-b0dead5202f3 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:23:46.881278 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-b89b0abb-aa22-4dd3-b7ef-b0dead5202f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:23:46.953972 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 466/915] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:23:46 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 15:23:46.964977 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89b0abb-aa22-4dd3-b7ef-b0dead5202f3 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:46.971837 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfcdc3b-c202-44c5-b8aa-cac5b8f44788 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:47.008519 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470760ac-3170-41b0-93fb-08037d481930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:47.009199 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470760ac-3170-41b0-93fb-08037d481930 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:47.009487 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470760ac-3170-41b0-93fb-08037d481930 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:47.009767 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470760ac-3170-41b0-93fb-08037d481930 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:47.010523 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470760ac-3170-41b0-93fb-08037d481930 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:47.010830 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470760ac-3170-41b0-93fb-08037d481930 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:47.012479 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470760ac-3170-41b0-93fb-08037d481930 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:47.026904 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 450/916] 10.222.0.99 () {56 vars in 1115 bytes} [Thu Jul 14 15:23:46 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:47.026904 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470760ac-3170-41b0-93fb-08037d481930 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:47.029191 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:47.101359 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89b0abb-aa22-4dd3-b7ef-b0dead5202f3 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:47.108560 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:47.135368 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470760ac-3170-41b0-93fb-08037d481930 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:47.141880 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c52b5c4-8a4d-4da4-93ae-dae3b9416af6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:47.142556 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c52b5c4-8a4d-4da4-93ae-dae3b9416af6 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:47.142845 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c52b5c4-8a4d-4da4-93ae-dae3b9416af6 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:47.143125 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c52b5c4-8a4d-4da4-93ae-dae3b9416af6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:47.143970 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c52b5c4-8a4d-4da4-93ae-dae3b9416af6 None admin] RBAC: Adding query filter params (interface=public, service_id=a294f671816e49ea844a8ce4183c18bb, region_id=RegionOne) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:47.144292 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c52b5c4-8a4d-4da4-93ae-dae3b9416af6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:47.145680 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c52b5c4-8a4d-4da4-93ae-dae3b9416af6 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:47.151512 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 467/917] 10.222.0.99 () {56 vars in 1251 bytes} [Thu Jul 14 15:23:46 2022] GET /identity/v3/endpoints?service_id=a294f671816e49ea844a8ce4183c18bb&interface=public®ion_id=RegionOne => generated 194 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:47.152189 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c52b5c4-8a4d-4da4-93ae-dae3b9416af6 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:47.179003 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbffaba-fd1f-44c9-9a22-71964f496d88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:47.179845 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbffaba-fd1f-44c9-9a22-71964f496d88 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:47.180137 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbffaba-fd1f-44c9-9a22-71964f496d88 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:47.180421 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbffaba-fd1f-44c9-9a22-71964f496d88 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:47.181088 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbffaba-fd1f-44c9-9a22-71964f496d88 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:47.182607 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbffaba-fd1f-44c9-9a22-71964f496d88 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:47.195545 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-dbbffaba-fd1f-44c9-9a22-71964f496d88 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 14 15:23:47.195545 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:47.195545 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 15:23:47.195545 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 15:23:47.195545 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 15:23:47.195545 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 15:23:47.195545 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 15:23:47.195545 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 15:23:47.195545 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 14 15:23:47.195545 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:23:47.195545 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 15:23:47.195545 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:23:47.195545 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:47.195545 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:47.195545 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:47.195545 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:47.195545 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:47.196616 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:47.196616 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:47.196616 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:47.196616 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:47.196616 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:47.196616 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:47.196616 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 15:23:47.196616 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 15:23:47.196616 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 15:23:47.196616 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 15:23:47.196616 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:47.196616 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:47.196616 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:47.196616 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:47.196616 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:47.196616 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:47.197447 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:47.197447 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:47.197447 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:47.197447 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:47.197447 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:47.197447 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:47.197447 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:47.197447 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:47.197447 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:47.197447 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:47.197447 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 15:23:47.197447 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 15:23:47.197447 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 14 15:23:47.197447 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:23:47.201799 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 451/918] 10.222.0.99 () {56 vars in 1114 bytes} [Thu Jul 14 15:23:47 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:47.203061 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbffaba-fd1f-44c9-9a22-71964f496d88 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:47.269607 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c52b5c4-8a4d-4da4-93ae-dae3b9416af6 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:47.271586 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:47.309719 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbffaba-fd1f-44c9-9a22-71964f496d88 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:47.313624 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c87cd45-1279-40a1-91d0-519dc0aeb771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:47.314423 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c87cd45-1279-40a1-91d0-519dc0aeb771 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:47.314716 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c87cd45-1279-40a1-91d0-519dc0aeb771 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:47.314995 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c87cd45-1279-40a1-91d0-519dc0aeb771 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:47.325393 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c87cd45-1279-40a1-91d0-519dc0aeb771 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:47.326854 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c87cd45-1279-40a1-91d0-519dc0aeb771 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:47.344710 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-0c87cd45-1279-40a1-91d0-519dc0aeb771 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:23:47.344710 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:47.344710 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:47.344710 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:47.344710 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:47.344710 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:47.344710 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:47.344710 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:47.344710 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:47.344710 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:47.344710 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:47.344710 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:47.344710 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:23:47.344710 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:23:47.344710 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:23:47.345624 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:23:47.345624 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:47.345624 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:47.345624 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:47.345624 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:47.345624 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:47.345624 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:47.345624 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:47.345624 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:47.345624 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:47.345624 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:47.345624 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:47.345624 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:47.345624 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:47.345624 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:47.345624 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:47.345624 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:47.346498 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:23:47.346498 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:23:47.346498 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:23:47.346498 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:23:47.346498 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 468/919] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:23:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:47.346766 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c87cd45-1279-40a1-91d0-519dc0aeb771 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:47.352186 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfc4373-603f-44be-9b3e-49756b2da155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:47.352809 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfc4373-603f-44be-9b3e-49756b2da155 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:47.353097 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfc4373-603f-44be-9b3e-49756b2da155 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:47.353382 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfc4373-603f-44be-9b3e-49756b2da155 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:47.354076 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfc4373-603f-44be-9b3e-49756b2da155 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:47.355445 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfc4373-603f-44be-9b3e-49756b2da155 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:47.371907 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 452/920] 10.222.0.99 () {56 vars in 1087 bytes} [Thu Jul 14 15:23:47 2022] GET /identity/v3/services => generated 1497 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:23:47.371907 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfc4373-603f-44be-9b3e-49756b2da155 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:47.374246 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:47.482510 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c87cd45-1279-40a1-91d0-519dc0aeb771 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:47.522932 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfc4373-603f-44be-9b3e-49756b2da155 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:47.524990 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42622fa-f580-4641-b0f4-bce54d069be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:47.525992 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42622fa-f580-4641-b0f4-bce54d069be3 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:47.526504 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42622fa-f580-4641-b0f4-bce54d069be3 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:47.526983 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42622fa-f580-4641-b0f4-bce54d069be3 None admin] PATH_INFO: `/v3/services` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:47.527949 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42622fa-f580-4641-b0f4-bce54d069be3 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:47.528485 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42622fa-f580-4641-b0f4-bce54d069be3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:47.530093 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42622fa-f580-4641-b0f4-bce54d069be3 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:47.540177 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 469/921] 10.222.0.99 () {56 vars in 1114 bytes} [Thu Jul 14 15:23:47 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:47.542514 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42622fa-f580-4641-b0f4-bce54d069be3 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:47.565299 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42941f4c-b71b-4f21-bccb-684838b6e052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:47.565976 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42941f4c-b71b-4f21-bccb-684838b6e052 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:47.566481 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:47.566598 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42941f4c-b71b-4f21-bccb-684838b6e052 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:47.566598 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42941f4c-b71b-4f21-bccb-684838b6e052 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:47.567255 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42941f4c-b71b-4f21-bccb-684838b6e052 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:47.567556 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42941f4c-b71b-4f21-bccb-684838b6e052 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:47.569220 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42941f4c-b71b-4f21-bccb-684838b6e052 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:47.584378 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 453/922] 10.222.0.99 () {56 vars in 1109 bytes} [Thu Jul 14 15:23:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:47.592630 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42941f4c-b71b-4f21-bccb-684838b6e052 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:47.666803 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:47.674751 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42622fa-f580-4641-b0f4-bce54d069be3 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:47.708633 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42941f4c-b71b-4f21-bccb-684838b6e052 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:47.715373 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec476a0-af2b-4776-8ba0-75cafca13e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:47.716064 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec476a0-af2b-4776-8ba0-75cafca13e8d None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:47.716616 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec476a0-af2b-4776-8ba0-75cafca13e8d None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:47.716616 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec476a0-af2b-4776-8ba0-75cafca13e8d None admin] PATH_INFO: `/v3/services` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:47.717314 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec476a0-af2b-4776-8ba0-75cafca13e8d None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:47.717615 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec476a0-af2b-4776-8ba0-75cafca13e8d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:47.719058 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec476a0-af2b-4776-8ba0-75cafca13e8d None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:47.728975 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 470/923] 10.222.0.99 () {56 vars in 1114 bytes} [Thu Jul 14 15:23:47 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:23:47.728975 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fd6e17-dbe1-4104-8bb5-6fd9978fc9c0 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:47.728975 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fd6e17-dbe1-4104-8bb5-6fd9978fc9c0 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:47.728975 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fd6e17-dbe1-4104-8bb5-6fd9978fc9c0 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:47.728975 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 471/924] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:23:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:23:47.732527 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fd6e17-dbe1-4104-8bb5-6fd9978fc9c0 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:47.750687 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c42961-d4ec-496d-a855-8128253f5e49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:47.751584 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c42961-d4ec-496d-a855-8128253f5e49 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:47.751821 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c42961-d4ec-496d-a855-8128253f5e49 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:47.752102 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c42961-d4ec-496d-a855-8128253f5e49 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:47.759712 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c42961-d4ec-496d-a855-8128253f5e49 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:47.761406 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c42961-d4ec-496d-a855-8128253f5e49 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:47.778315 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-b9c42961-d4ec-496d-a855-8128253f5e49 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 15:23:47.778315 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:47.778315 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:47.778315 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:47.778315 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:47.778315 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:47.778315 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:47.778315 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:47.778315 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:47.778315 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:47.778315 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:47.778315 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:47.778315 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 15:23:47.778315 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 15:23:47.778315 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 15:23:47.778315 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 15:23:47.779492 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:47.779492 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:47.779492 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 15:23:47.779492 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:23:47.779492 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 15:23:47.779492 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:23:47.779492 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:47.779492 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:47.779492 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:47.779492 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:47.779492 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:47.779492 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:47.779492 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:47.779492 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:47.779492 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:47.779492 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:47.779492 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:47.779492 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:47.780513 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:47.780513 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:47.780513 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 15:23:47.780513 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 15:23:47.780513 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 15:23:47.780513 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 15:23:47.780513 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 15:23:47.780513 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 15:23:47.780513 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 15:23:47.780513 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:23:47.780513 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 454/925] 10.222.0.99 () {56 vars in 1117 bytes} [Thu Jul 14 15:23:47 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:47.781357 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f82f377-0a8d-4430-acf9-a205de4f69a8 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:47.781608 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f82f377-0a8d-4430-acf9-a205de4f69a8 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:47.781886 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f82f377-0a8d-4430-acf9-a205de4f69a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:47.800463 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-9f82f377-0a8d-4430-acf9-a205de4f69a8 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:23:47.801670 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-9f82f377-0a8d-4430-acf9-a205de4f69a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:23:47.829890 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:47.851894 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fd6e17-dbe1-4104-8bb5-6fd9978fc9c0 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:47.877744 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 455/926] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:23:47 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 15:23:47.878523 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f82f377-0a8d-4430-acf9-a205de4f69a8 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:47.904683 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916c9b05-5f76-4013-848e-4b64cc642db1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:47.905366 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916c9b05-5f76-4013-848e-4b64cc642db1 None admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:47.905653 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916c9b05-5f76-4013-848e-4b64cc642db1 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:47.906052 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916c9b05-5f76-4013-848e-4b64cc642db1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:47.906786 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916c9b05-5f76-4013-848e-4b64cc642db1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:47.908199 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916c9b05-5f76-4013-848e-4b64cc642db1 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:47.930335 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 472/927] 10.222.0.99 () {60 vars in 1144 bytes} [Thu Jul 14 15:23:47 2022] POST /identity/v3/endpoints => generated 346 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:23:47.945423 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916c9b05-5f76-4013-848e-4b64cc642db1 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:47.948122 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:48.015068 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f82f377-0a8d-4430-acf9-a205de4f69a8 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:48.068211 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f7baf2-77be-496b-aec7-a3c61e428c1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:48.069618 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f7baf2-77be-496b-aec7-a3c61e428c1e None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:48.070005 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f7baf2-77be-496b-aec7-a3c61e428c1e None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:48.070378 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f7baf2-77be-496b-aec7-a3c61e428c1e None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:48.071795 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f7baf2-77be-496b-aec7-a3c61e428c1e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:48.072210 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f7baf2-77be-496b-aec7-a3c61e428c1e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:48.074087 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f7baf2-77be-496b-aec7-a3c61e428c1e None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:48.099610 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 456/928] 10.222.0.99 () {56 vars in 1149 bytes} [Thu Jul 14 15:23:47 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:48.100727 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916c9b05-5f76-4013-848e-4b64cc642db1 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:48.101211 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f7baf2-77be-496b-aec7-a3c61e428c1e None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:48.148422 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cf36f0-df80-4f95-a97f-9c28f66df176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:48.149680 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cf36f0-df80-4f95-a97f-9c28f66df176 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:48.150207 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cf36f0-df80-4f95-a97f-9c28f66df176 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:48.150531 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cf36f0-df80-4f95-a97f-9c28f66df176 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:48.151228 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cf36f0-df80-4f95-a97f-9c28f66df176 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:48.153091 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cf36f0-df80-4f95-a97f-9c28f66df176 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:48.168880 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-56cf36f0-df80-4f95-a97f-9c28f66df176 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 15:23:48.168880 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:48.168880 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 15:23:48.168880 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 15:23:48.168880 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 15:23:48.168880 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 15:23:48.168880 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 15:23:48.168880 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 15:23:48.168880 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 15:23:48.168880 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:23:48.168880 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 15:23:48.168880 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:23:48.168880 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:48.168880 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:48.168880 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:48.168880 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:48.168880 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:48.170067 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:48.170067 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:48.170067 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:48.170067 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:48.170067 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:48.170067 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:48.170067 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 15:23:48.170067 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 15:23:48.170067 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 15:23:48.170067 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 15:23:48.170067 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:48.170067 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:48.170067 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:48.170067 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:48.170067 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:48.170067 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:48.170895 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:48.170895 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:48.170895 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:48.170895 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:48.170895 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:48.170895 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:48.170895 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:48.170895 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:48.170895 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:48.170895 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:48.170895 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 15:23:48.170895 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 15:23:48.170895 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 15:23:48.170895 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:23:48.170895 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 473/929] 10.222.0.99 () {56 vars in 1126 bytes} [Thu Jul 14 15:23:47 2022] GET /identity/v3/services/object-store => generated 93 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:48.177202 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cf36f0-df80-4f95-a97f-9c28f66df176 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:48.180283 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:48.236459 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f7baf2-77be-496b-aec7-a3c61e428c1e None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:48.255988 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:48.286079 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68852bc5-2a47-407d-8f8f-8f9e4259aee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:48.287171 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68852bc5-2a47-407d-8f8f-8f9e4259aee4 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:48.287644 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68852bc5-2a47-407d-8f8f-8f9e4259aee4 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:48.288075 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68852bc5-2a47-407d-8f8f-8f9e4259aee4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:48.303985 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68852bc5-2a47-407d-8f8f-8f9e4259aee4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:48.305199 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cf36f0-df80-4f95-a97f-9c28f66df176 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:48.306263 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68852bc5-2a47-407d-8f8f-8f9e4259aee4 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:48.330789 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-68852bc5-2a47-407d-8f8f-8f9e4259aee4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:23:48.330789 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:48.330789 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:48.330789 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:48.330789 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:48.330789 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:48.330789 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:48.330789 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:48.330789 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:48.330789 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:48.330789 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:48.330789 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:48.330789 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:23:48.330789 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:23:48.330789 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:23:48.331812 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:23:48.331812 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:48.331812 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:48.331812 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:48.331812 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:48.331812 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:48.331812 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:48.331812 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:48.331812 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:48.331812 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:48.331812 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:48.331812 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:48.331812 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:48.331812 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:48.331812 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:48.331812 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:48.331812 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:48.332703 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:23:48.332703 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:23:48.332703 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:23:48.332703 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:23:48.333703 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 457/930] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:23:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 15:23:48.343824 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68852bc5-2a47-407d-8f8f-8f9e4259aee4 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:48.350115 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5603877a-fd8d-4640-a593-f1909a63b4af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:48.350801 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5603877a-fd8d-4640-a593-f1909a63b4af None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:48.351090 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5603877a-fd8d-4640-a593-f1909a63b4af None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:48.351372 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5603877a-fd8d-4640-a593-f1909a63b4af None admin] PATH_INFO: `/v3/services` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:48.352130 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5603877a-fd8d-4640-a593-f1909a63b4af None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:48.352441 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5603877a-fd8d-4640-a593-f1909a63b4af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:48.353830 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5603877a-fd8d-4640-a593-f1909a63b4af None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:48.360373 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 474/931] 10.222.0.99 () {56 vars in 1122 bytes} [Thu Jul 14 15:23:48 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:23:48.366368 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5603877a-fd8d-4640-a593-f1909a63b4af None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:48.444768 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:48.455697 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68852bc5-2a47-407d-8f8f-8f9e4259aee4 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:48.487941 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5603877a-fd8d-4640-a593-f1909a63b4af None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:48.497217 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c7d860-7b83-404b-b57f-1ab5a7e04df5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:48.498311 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c7d860-7b83-404b-b57f-1ab5a7e04df5 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:48.498311 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c7d860-7b83-404b-b57f-1ab5a7e04df5 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:48.498595 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c7d860-7b83-404b-b57f-1ab5a7e04df5 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:48.499328 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c7d860-7b83-404b-b57f-1ab5a7e04df5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:48.499707 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c7d860-7b83-404b-b57f-1ab5a7e04df5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:48.501591 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c7d860-7b83-404b-b57f-1ab5a7e04df5 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:48.519921 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 458/932] 10.222.0.99 () {56 vars in 1109 bytes} [Thu Jul 14 15:23:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:48.521130 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c7d860-7b83-404b-b57f-1ab5a7e04df5 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:48.532123 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85c6831-61ee-4100-bef6-b7e6302e4c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:48.533250 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85c6831-61ee-4100-bef6-b7e6302e4c45 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:48.533785 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85c6831-61ee-4100-bef6-b7e6302e4c45 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:48.534317 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85c6831-61ee-4100-bef6-b7e6302e4c45 None admin] PATH_INFO: `/v3/services` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:48.535266 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85c6831-61ee-4100-bef6-b7e6302e4c45 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:48.535866 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85c6831-61ee-4100-bef6-b7e6302e4c45 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:48.537526 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85c6831-61ee-4100-bef6-b7e6302e4c45 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:48.547697 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 475/933] 10.222.0.99 () {56 vars in 1122 bytes} [Thu Jul 14 15:23:48 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:23:48.549929 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:48.550828 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85c6831-61ee-4100-bef6-b7e6302e4c45 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:48.636022 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c7d860-7b83-404b-b57f-1ab5a7e04df5 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:48.665840 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85c6831-61ee-4100-bef6-b7e6302e4c45 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:48.674586 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60f5a15-9279-430e-b48a-25248037c0fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:48.675684 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60f5a15-9279-430e-b48a-25248037c0fc None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:48.675812 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60f5a15-9279-430e-b48a-25248037c0fc None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:48.676064 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60f5a15-9279-430e-b48a-25248037c0fc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:48.684307 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60f5a15-9279-430e-b48a-25248037c0fc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:48.686064 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60f5a15-9279-430e-b48a-25248037c0fc None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:48.700338 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:48.702623 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-e60f5a15-9279-430e-b48a-25248037c0fc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:23:48.702623 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:48.702623 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:48.702623 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:48.702623 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:48.702623 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:48.702623 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:48.702623 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:48.702623 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:48.702623 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:48.702623 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:48.702623 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:48.702623 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 15:23:48.702623 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 15:23:48.702623 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 15:23:48.704641 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 15:23:48.704641 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:48.704641 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:48.704641 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:48.704641 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:48.704641 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:48.704641 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:48.704641 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:48.704641 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:48.704641 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:48.704641 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:48.704641 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:48.704641 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:48.704641 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:48.704641 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:48.704641 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:48.704641 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:48.705554 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 15:23:48.705554 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 15:23:48.705554 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 15:23:48.705554 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 15:23:48.705554 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 15:23:48.705554 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 15:23:48.705554 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:23:48.705554 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:23:48.707980 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 459/934] 10.222.0.99 () {56 vars in 1111 bytes} [Thu Jul 14 15:23:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:48.711273 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60f5a15-9279-430e-b48a-25248037c0fc None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:48.712433 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c563919-5601-4cb8-9298-69dfcde1a4ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:48.713359 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c563919-5601-4cb8-9298-69dfcde1a4ef None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:48.713880 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c563919-5601-4cb8-9298-69dfcde1a4ef None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:48.714364 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c563919-5601-4cb8-9298-69dfcde1a4ef None admin] PATH_INFO: `/v3/endpoints` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:48.715386 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c563919-5601-4cb8-9298-69dfcde1a4ef None admin] RBAC: Adding query filter params (interface=public, service_id=079d217a520c42e2b41b996c39af0794, region_id=RegionOne) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:48.715984 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c563919-5601-4cb8-9298-69dfcde1a4ef None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:48.717641 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c563919-5601-4cb8-9298-69dfcde1a4ef None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:48.729817 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 476/935] 10.222.0.99 () {56 vars in 1251 bytes} [Thu Jul 14 15:23:48 2022] GET /identity/v3/endpoints?service_id=079d217a520c42e2b41b996c39af0794&interface=public®ion_id=RegionOne => generated 194 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:23:48.731158 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c563919-5601-4cb8-9298-69dfcde1a4ef None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:48.805529 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:48.827574 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60f5a15-9279-430e-b48a-25248037c0fc None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:48.853670 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c563919-5601-4cb8-9298-69dfcde1a4ef None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:48.865983 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb293fd-4c20-4148-9dc8-776b784ce2d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:48.866662 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb293fd-4c20-4148-9dc8-776b784ce2d7 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:48.866948 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb293fd-4c20-4148-9dc8-776b784ce2d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:48.867226 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb293fd-4c20-4148-9dc8-776b784ce2d7 None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:48.868043 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb293fd-4c20-4148-9dc8-776b784ce2d7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:48.868351 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb293fd-4c20-4148-9dc8-776b784ce2d7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:48.869873 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb293fd-4c20-4148-9dc8-776b784ce2d7 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:48.888422 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 460/936] 10.222.0.99 () {56 vars in 1148 bytes} [Thu Jul 14 15:23:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:23:48.889279 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb293fd-4c20-4148-9dc8-776b784ce2d7 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:48.894131 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64833367-682f-4c3a-89be-c69f33c55b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:48.895096 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64833367-682f-4c3a-89be-c69f33c55b3e None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:48.895589 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64833367-682f-4c3a-89be-c69f33c55b3e None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:48.896137 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64833367-682f-4c3a-89be-c69f33c55b3e None admin] PATH_INFO: `/v3/services` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:48.897082 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64833367-682f-4c3a-89be-c69f33c55b3e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:48.898778 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64833367-682f-4c3a-89be-c69f33c55b3e None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:48.911980 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 477/937] 10.222.0.99 () {56 vars in 1087 bytes} [Thu Jul 14 15:23:48 2022] GET /identity/v3/services => generated 1497 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:23:49.013764 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb293fd-4c20-4148-9dc8-776b784ce2d7 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:49.060675 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22519780-af7c-4b03-9370-92dad3dadb43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:49.061547 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22519780-af7c-4b03-9370-92dad3dadb43 None admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:49.061836 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22519780-af7c-4b03-9370-92dad3dadb43 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:49.062891 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22519780-af7c-4b03-9370-92dad3dadb43 None admin] PATH_INFO: `/v3/projects/9048d1a6f00443ec92f11471babe5a56/users/adaeadf613954d7393b3fd3bae68c362/roles/ebca1dcd25d64cebaae5f4c18285be4a` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:49.066914 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22519780-af7c-4b03-9370-92dad3dadb43 None admin] RBAC: Authorizing `identity:create_grant(project_id=9048d1a6f00443ec92f11471babe5a56, user_id=adaeadf613954d7393b3fd3bae68c362, role_id=ebca1dcd25d64cebaae5f4c18285be4a)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:49.069335 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22519780-af7c-4b03-9370-92dad3dadb43 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:49.081314 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:49.087131 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 461/938] 10.222.0.99 () {58 vars in 1439 bytes} [Thu Jul 14 15:23:48 2022] PUT /identity/v3/projects/9048d1a6f00443ec92f11471babe5a56/users/adaeadf613954d7393b3fd3bae68c362/roles/ebca1dcd25d64cebaae5f4c18285be4a => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:23:49.579426 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b4b305-c8f4-408c-9e05-31710e915b4d None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:49.580480 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b4b305-c8f4-408c-9e05-31710e915b4d None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:49.580973 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b4b305-c8f4-408c-9e05-31710e915b4d None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:49.582861 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 478/939] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:23:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:23:49.588528 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300789c1-bdd1-40ab-9243-b7a097553aa6 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:49.589387 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300789c1-bdd1-40ab-9243-b7a097553aa6 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:49.589872 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300789c1-bdd1-40ab-9243-b7a097553aa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:49.608361 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-300789c1-bdd1-40ab-9243-b7a097553aa6 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:23:49.609996 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-300789c1-bdd1-40ab-9243-b7a097553aa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:23:49.687801 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 462/940] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:23:49 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 15:23:49.704399 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b4b305-c8f4-408c-9e05-31710e915b4d None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:49.766394 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:49.813292 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b4b305-c8f4-408c-9e05-31710e915b4d None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:49.848984 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815fcfba-dd2a-4ca1-9516-d55f3726941b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:49.849726 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815fcfba-dd2a-4ca1-9516-d55f3726941b None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:49.850017 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815fcfba-dd2a-4ca1-9516-d55f3726941b None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:49.850297 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815fcfba-dd2a-4ca1-9516-d55f3726941b None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:49.850960 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815fcfba-dd2a-4ca1-9516-d55f3726941b None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:49.852381 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815fcfba-dd2a-4ca1-9516-d55f3726941b None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:49.865904 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-815fcfba-dd2a-4ca1-9516-d55f3726941b None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 14 15:23:49.865904 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:49.865904 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 15:23:49.865904 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 15:23:49.865904 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 15:23:49.865904 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 15:23:49.865904 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 15:23:49.865904 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 15:23:49.865904 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 14 15:23:49.865904 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:23:49.865904 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 15:23:49.865904 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:23:49.865904 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:49.865904 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:49.865904 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:49.865904 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:49.866859 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:49.866859 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:49.866859 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:49.866859 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:49.866859 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:49.866859 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:49.866859 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:49.866859 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 15:23:49.866859 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 15:23:49.866859 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 15:23:49.866859 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 15:23:49.866859 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:49.866859 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:49.866859 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:49.866859 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:49.866859 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:49.867713 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:49.867713 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:49.867713 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:49.867713 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:49.867713 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:49.867713 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:49.867713 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:49.867713 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:49.867713 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:49.867713 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:49.867713 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:49.867713 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 15:23:49.867713 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 15:23:49.867713 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 14 15:23:49.867713 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:23:49.867713 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 479/941] 10.222.0.99 () {56 vars in 1132 bytes} [Thu Jul 14 15:23:49 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:49.873229 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300789c1-bdd1-40ab-9243-b7a097553aa6 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:49.974782 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300789c1-bdd1-40ab-9243-b7a097553aa6 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:50.005997 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:50.013362 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed4e82e-9729-42d1-8840-20e77bb2bf91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:50.014047 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed4e82e-9729-42d1-8840-20e77bb2bf91 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:50.014334 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed4e82e-9729-42d1-8840-20e77bb2bf91 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:50.014610 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed4e82e-9729-42d1-8840-20e77bb2bf91 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:50.015457 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed4e82e-9729-42d1-8840-20e77bb2bf91 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:50.015813 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed4e82e-9729-42d1-8840-20e77bb2bf91 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:50.017192 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed4e82e-9729-42d1-8840-20e77bb2bf91 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:50.026979 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 463/942] 10.222.0.99 () {56 vars in 1126 bytes} [Thu Jul 14 15:23:49 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:50.028316 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815fcfba-dd2a-4ca1-9516-d55f3726941b None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:50.116186 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8c6272-e4e5-4684-bb2f-7da2cf950acc None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:50.116633 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8c6272-e4e5-4684-bb2f-7da2cf950acc None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:50.117239 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8c6272-e4e5-4684-bb2f-7da2cf950acc None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:50.119692 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 464/943] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:23:50 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:23:50.130004 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c195752-59de-4c89-bcca-6891719d9085 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:50.130433 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c195752-59de-4c89-bcca-6891719d9085 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:50.130900 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c195752-59de-4c89-bcca-6891719d9085 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:50.138042 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815fcfba-dd2a-4ca1-9516-d55f3726941b None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:50.150160 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-1c195752-59de-4c89-bcca-6891719d9085 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:23:50.151746 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-1c195752-59de-4c89-bcca-6891719d9085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:23:50.163144 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:50.182839 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda9ad82-ef5d-4da1-83f7-8daaa329f257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:50.184653 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda9ad82-ef5d-4da1-83f7-8daaa329f257 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:50.185396 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda9ad82-ef5d-4da1-83f7-8daaa329f257 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:50.185892 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda9ad82-ef5d-4da1-83f7-8daaa329f257 None admin] PATH_INFO: `/v3/services` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:50.187096 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda9ad82-ef5d-4da1-83f7-8daaa329f257 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:50.188260 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda9ad82-ef5d-4da1-83f7-8daaa329f257 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:50.189738 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda9ad82-ef5d-4da1-83f7-8daaa329f257 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:50.199630 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 480/944] 10.222.0.99 () {56 vars in 1126 bytes} [Thu Jul 14 15:23:50 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:50.204025 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda9ad82-ef5d-4da1-83f7-8daaa329f257 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:50.255677 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 465/945] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:23:50 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 15:23:50.279203 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c195752-59de-4c89-bcca-6891719d9085 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:50.337983 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda9ad82-ef5d-4da1-83f7-8daaa329f257 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:50.384421 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5469a4c-38fa-4895-ab1d-30f47e9dd744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:50.386040 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5469a4c-38fa-4895-ab1d-30f47e9dd744 None admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:50.386780 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5469a4c-38fa-4895-ab1d-30f47e9dd744 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:50.387514 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5469a4c-38fa-4895-ab1d-30f47e9dd744 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:50.388920 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5469a4c-38fa-4895-ab1d-30f47e9dd744 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:50.391028 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5469a4c-38fa-4895-ab1d-30f47e9dd744 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:50.394785 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c195752-59de-4c89-bcca-6891719d9085 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:50.424210 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 481/946] 10.222.0.99 () {60 vars in 1144 bytes} [Thu Jul 14 15:23:50 2022] POST /identity/v3/endpoints => generated 347 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:23:50.431117 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:50.451790 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8343ea13-ec7f-4bf3-897f-fc7fa87cf26f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:50.452476 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8343ea13-ec7f-4bf3-897f-fc7fa87cf26f None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:50.452766 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8343ea13-ec7f-4bf3-897f-fc7fa87cf26f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:50.453618 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8343ea13-ec7f-4bf3-897f-fc7fa87cf26f None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:50.454346 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8343ea13-ec7f-4bf3-897f-fc7fa87cf26f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:50.455874 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8343ea13-ec7f-4bf3-897f-fc7fa87cf26f None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:50.457375 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: WARNING keystone.server.flask.common [None req-8343ea13-ec7f-4bf3-897f-fc7fa87cf26f None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 14 15:23:50.478853 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.sql.core [None req-8343ea13-ec7f-4bf3-897f-fc7fa87cf26f None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 14 15:23:50.478853 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 14 15:23:50.478853 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [parameters: {'id': '58373c45001943129aa086c9a55e9c74', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 14 15:23:50.478853 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80237) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 14 15:23:50.488069 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-8343ea13-ec7f-4bf3-897f-fc7fa87cf26f None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 58373c45001943129aa086c9a55e9c74.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 58373c45001943129aa086c9a55e9c74. Jul 14 15:23:50.488069 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:50.488069 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 14 15:23:50.488069 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 14 15:23:50.488069 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 14 15:23:50.488069 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 14 15:23:50.488069 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 14 15:23:50.488069 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application result = self._query(query) Jul 14 15:23:50.488069 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 14 15:23:50.488069 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application conn.query(q) Jul 14 15:23:50.488069 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 14 15:23:50.488069 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 14 15:23:50.488069 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 14 15:23:50.488948 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application result.read() Jul 14 15:23:50.488948 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 14 15:23:50.488948 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 14 15:23:50.488948 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 14 15:23:50.488948 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 14 15:23:50.488948 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 14 15:23:50.488948 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 14 15:23:50.488948 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 14 15:23:50.488948 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 14 15:23:50.488948 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 14 15:23:50.488948 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:23:50.488948 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 14 15:23:50.488948 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:23:50.488948 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:50.488948 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 14 15:23:50.488948 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 14 15:23:50.488948 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 14 15:23:50.488948 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 14 15:23:50.490358 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 14 15:23:50.490358 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application next(self.gen) Jul 14 15:23:50.490358 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 14 15:23:50.490358 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application yield resource Jul 14 15:23:50.490358 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 14 15:23:50.490358 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application next(self.gen) Jul 14 15:23:50.490358 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 14 15:23:50.490358 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self.session.rollback() Jul 14 15:23:50.490358 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 14 15:23:50.490358 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self.force_reraise() Jul 14 15:23:50.490358 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 14 15:23:50.490358 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise self.value Jul 14 15:23:50.490358 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 14 15:23:50.490358 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 14 15:23:50.490358 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 14 15:23:50.490358 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application session.commit() Jul 14 15:23:50.490358 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 14 15:23:50.491549 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 14 15:23:50.491549 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 14 15:23:50.491549 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._prepare_impl() Jul 14 15:23:50.491549 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 14 15:23:50.491549 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self.session.flush() Jul 14 15:23:50.491549 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 14 15:23:50.491549 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._flush(objects) Jul 14 15:23:50.491549 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 14 15:23:50.491549 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 14 15:23:50.491549 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 14 15:23:50.491549 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application compat.raise_( Jul 14 15:23:50.491549 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 14 15:23:50.491549 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception Jul 14 15:23:50.491549 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 14 15:23:50.491549 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application flush_context.execute() Jul 14 15:23:50.491549 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 14 15:23:50.491549 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rec.execute(self) Jul 14 15:23:50.492496 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 14 15:23:50.492496 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 14 15:23:50.492496 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 14 15:23:50.492496 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 14 15:23:50.492496 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 14 15:23:50.492496 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 14 15:23:50.492496 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 14 15:23:50.492496 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 14 15:23:50.492496 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 14 15:23:50.492496 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 14 15:23:50.492496 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 14 15:23:50.492496 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 14 15:23:50.492496 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 14 15:23:50.492496 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 14 15:23:50.492496 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 14 15:23:50.493304 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 14 15:23:50.493304 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 14 15:23:50.493304 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception Jul 14 15:23:50.493304 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 14 15:23:50.493304 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 14 15:23:50.493304 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 14 15:23:50.493304 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 14 15:23:50.493304 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 14 15:23:50.493304 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application result = self._query(query) Jul 14 15:23:50.493304 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 14 15:23:50.493304 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application conn.query(q) Jul 14 15:23:50.493304 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 14 15:23:50.493304 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 14 15:23:50.493304 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 14 15:23:50.493304 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application result.read() Jul 14 15:23:50.493304 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 14 15:23:50.493304 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 14 15:23:50.494191 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 14 15:23:50.494191 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 14 15:23:50.494191 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 14 15:23:50.494191 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 14 15:23:50.494191 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 14 15:23:50.494191 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 14 15:23:50.494191 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 14 15:23:50.494191 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 14 15:23:50.494191 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [parameters: {'id': '58373c45001943129aa086c9a55e9c74', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 14 15:23:50.494191 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 14 15:23:50.494191 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:23:50.494191 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 15:23:50.494191 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:23:50.494191 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:50.494191 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 14 15:23:50.496451 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 14 15:23:50.496451 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 14 15:23:50.496451 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 14 15:23:50.496451 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 14 15:23:50.496451 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:23:50.496451 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 15:23:50.496451 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:23:50.496451 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:50.496451 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:50.496451 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:50.496451 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:50.496451 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:50.496451 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:50.496451 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:50.496451 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:50.496451 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:50.496451 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:50.498552 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:50.498552 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 14 15:23:50.498552 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 14 15:23:50.498552 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:50.498552 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:50.498552 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 14 15:23:50.498552 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 14 15:23:50.498552 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 58373c45001943129aa086c9a55e9c74. Jul 14 15:23:50.498552 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:23:50.498552 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 466/947] 10.222.0.99 () {60 vars in 1140 bytes} [Thu Jul 14 15:23:50 2022] POST /identity/v3/projects => generated 262 bytes in 211 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 14 15:23:50.503913 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5469a4c-38fa-4895-ab1d-30f47e9dd744 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:50.552533 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:50.619788 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5469a4c-38fa-4895-ab1d-30f47e9dd744 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:50.658602 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff10edc-43f4-46db-b96d-05503e559ce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:50.659434 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff10edc-43f4-46db-b96d-05503e559ce9 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:50.659758 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff10edc-43f4-46db-b96d-05503e559ce9 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:50.660042 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff10edc-43f4-46db-b96d-05503e559ce9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:50.667295 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff10edc-43f4-46db-b96d-05503e559ce9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:50.668806 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff10edc-43f4-46db-b96d-05503e559ce9 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:50.685636 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-cff10edc-43f4-46db-b96d-05503e559ce9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:23:50.685636 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:50.685636 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:50.685636 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:50.685636 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:50.685636 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:50.685636 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:50.685636 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:50.685636 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:50.685636 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:50.685636 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:50.685636 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:50.685636 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 15:23:50.685636 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 15:23:50.685636 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 15:23:50.686573 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 15:23:50.686573 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:50.686573 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:50.686573 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:50.686573 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:50.686573 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:50.686573 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:50.686573 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:50.686573 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:50.686573 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:50.686573 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:50.686573 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:50.686573 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:50.686573 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:50.686573 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:50.686573 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:50.686573 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:50.688106 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 15:23:50.688106 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 15:23:50.688106 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 15:23:50.688106 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 15:23:50.688106 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 15:23:50.688106 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 15:23:50.688106 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:23:50.688106 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:23:50.688106 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 482/948] 10.222.0.99 () {56 vars in 1111 bytes} [Thu Jul 14 15:23:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:50.693436 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8343ea13-ec7f-4bf3-897f-fc7fa87cf26f None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:50.800837 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8343ea13-ec7f-4bf3-897f-fc7fa87cf26f None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:50.837485 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6133acc8-ebab-4e8e-8852-5526eeef6d1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:50.838531 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6133acc8-ebab-4e8e-8852-5526eeef6d1c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:50.839039 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6133acc8-ebab-4e8e-8852-5526eeef6d1c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:50.839519 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6133acc8-ebab-4e8e-8852-5526eeef6d1c None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:50.840521 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6133acc8-ebab-4e8e-8852-5526eeef6d1c None admin] RBAC: Adding query filter params (name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:50.841029 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6133acc8-ebab-4e8e-8852-5526eeef6d1c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:50.842864 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6133acc8-ebab-4e8e-8852-5526eeef6d1c None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:50.866249 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 467/949] 10.222.0.99 () {56 vars in 1112 bytes} [Thu Jul 14 15:23:50 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:51.169012 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fd0e5f-8612-43e8-ad95-350af9b8693a None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:51.169412 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fd0e5f-8612-43e8-ad95-350af9b8693a None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:51.169591 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fd0e5f-8612-43e8-ad95-350af9b8693a None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:51.171196 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 483/950] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:23:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:23:51.177903 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cc7e3d-8943-4839-b795-0158a9e74825 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:51.178193 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cc7e3d-8943-4839-b795-0158a9e74825 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:51.178476 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cc7e3d-8943-4839-b795-0158a9e74825 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:51.197838 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-c6cc7e3d-8943-4839-b795-0158a9e74825 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:23:51.199413 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c6cc7e3d-8943-4839-b795-0158a9e74825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:23:51.233224 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:51.284944 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 468/951] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:23:51 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 15:23:51.301980 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83265d3-ea28-49e5-b519-d3041a0b43cb None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:51.302279 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83265d3-ea28-49e5-b519-d3041a0b43cb None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:51.302545 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83265d3-ea28-49e5-b519-d3041a0b43cb None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:51.304199 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 484/952] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:23:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:23:51.308969 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83265d3-ea28-49e5-b519-d3041a0b43cb None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:51.309369 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5674b152-c79c-4432-b38f-2a0523203805 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:51.309669 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5674b152-c79c-4432-b38f-2a0523203805 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:51.309954 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5674b152-c79c-4432-b38f-2a0523203805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:51.327261 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-5674b152-c79c-4432-b38f-2a0523203805 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:23:51.328514 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-5674b152-c79c-4432-b38f-2a0523203805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:23:51.423709 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 469/953] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:23:51 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 15:23:51.440928 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5674b152-c79c-4432-b38f-2a0523203805 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:51.445044 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83265d3-ea28-49e5-b519-d3041a0b43cb None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:51.446661 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:51.495474 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb81a497-251e-4e47-b57d-6fda7946ba33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:51.496469 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb81a497-251e-4e47-b57d-6fda7946ba33 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:51.496761 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb81a497-251e-4e47-b57d-6fda7946ba33 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:51.497043 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb81a497-251e-4e47-b57d-6fda7946ba33 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:51.497982 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb81a497-251e-4e47-b57d-6fda7946ba33 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:51.500378 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb81a497-251e-4e47-b57d-6fda7946ba33 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:51.518526 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-eb81a497-251e-4e47-b57d-6fda7946ba33 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 15:23:51.518526 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:51.518526 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 15:23:51.518526 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 15:23:51.518526 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 15:23:51.518526 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 15:23:51.518526 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 15:23:51.518526 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 15:23:51.518526 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 15:23:51.518526 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:23:51.518526 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 15:23:51.518526 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:23:51.518526 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:51.518526 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:51.518526 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:51.518526 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:51.518526 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:51.519962 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:51.519962 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:51.519962 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:51.519962 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:51.519962 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:51.519962 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:51.519962 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 15:23:51.519962 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 15:23:51.519962 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 15:23:51.519962 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 15:23:51.519962 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:51.519962 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:51.519962 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:51.519962 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:51.519962 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:51.519962 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:51.521280 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:51.521280 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:51.521280 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:51.521280 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:51.521280 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:51.521280 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:51.521280 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:51.521280 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:51.521280 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:51.521280 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:51.521280 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 15:23:51.521280 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 15:23:51.521280 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 15:23:51.521280 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:23:51.523680 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 485/954] 10.222.0.99 () {56 vars in 1126 bytes} [Thu Jul 14 15:23:51 2022] GET /identity/v3/services/object-store => generated 93 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:51.528864 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb81a497-251e-4e47-b57d-6fda7946ba33 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:51.563689 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:51.571857 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5674b152-c79c-4432-b38f-2a0523203805 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:51.616662 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c075f5-f713-4e61-ab48-a66c96cc29d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:51.618577 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c075f5-f713-4e61-ab48-a66c96cc29d5 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:51.619057 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c075f5-f713-4e61-ab48-a66c96cc29d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:51.619525 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c075f5-f713-4e61-ab48-a66c96cc29d5 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:51.634414 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c075f5-f713-4e61-ab48-a66c96cc29d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:23:51.646554 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-48c075f5-f713-4e61-ab48-a66c96cc29d5 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 15:23:51.646554 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:51.646554 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:51.646554 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:51.646554 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:51.646554 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:51.646554 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:51.646554 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:51.646554 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:51.646554 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:51.646554 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:51.646554 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:51.646554 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:23:51.646554 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:23:51.646554 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:23:51.646554 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:23:51.647685 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:23:51.647685 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:23:51.647685 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:23:51.647685 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:23:51.647685 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:51.647685 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:51.647685 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:51.647685 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:51.647685 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:51.647685 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:51.647685 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:51.647685 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:51.647685 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:51.647685 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:51.647685 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:51.647685 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:51.648620 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:51.648620 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:51.648620 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:51.648620 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:51.648620 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:23:51.648620 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:23:51.648620 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:23:51.648620 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:23:51.648620 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:23:51.648620 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:23:51.648620 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 15:23:51.648620 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:23:51.651660 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 470/955] 10.222.0.99 () {56 vars in 1120 bytes} [Thu Jul 14 15:23:51 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:51.655811 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c075f5-f713-4e61-ab48-a66c96cc29d5 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:51.658035 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb81a497-251e-4e47-b57d-6fda7946ba33 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:51.701121 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1e1790-1dfc-466b-9884-38bcc8e5667c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:51.701894 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1e1790-1dfc-466b-9884-38bcc8e5667c None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:51.702179 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1e1790-1dfc-466b-9884-38bcc8e5667c None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:51.702459 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1e1790-1dfc-466b-9884-38bcc8e5667c None admin] PATH_INFO: `/v3/services` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:51.703825 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1e1790-1dfc-466b-9884-38bcc8e5667c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:51.704138 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1e1790-1dfc-466b-9884-38bcc8e5667c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:51.705791 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1e1790-1dfc-466b-9884-38bcc8e5667c None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:51.717703 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 486/956] 10.222.0.99 () {56 vars in 1122 bytes} [Thu Jul 14 15:23:51 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:51.718090 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1e1790-1dfc-466b-9884-38bcc8e5667c None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:51.753800 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:51.787524 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c075f5-f713-4e61-ab48-a66c96cc29d5 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:51.824489 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f77c9f-d1ee-4a0e-aa81-007203510afe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:51.825395 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f77c9f-d1ee-4a0e-aa81-007203510afe None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:51.825508 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f77c9f-d1ee-4a0e-aa81-007203510afe None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:51.825883 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f77c9f-d1ee-4a0e-aa81-007203510afe None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:51.826844 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f77c9f-d1ee-4a0e-aa81-007203510afe None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:51.827259 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f77c9f-d1ee-4a0e-aa81-007203510afe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:51.829239 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f77c9f-d1ee-4a0e-aa81-007203510afe None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:51.836038 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1e1790-1dfc-466b-9884-38bcc8e5667c None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:51.838596 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 471/957] 10.222.0.99 () {56 vars in 1115 bytes} [Thu Jul 14 15:23:51 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:51.847198 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f77c9f-d1ee-4a0e-aa81-007203510afe None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:51.873681 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:51.890968 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c36e38-b258-4de6-bccb-40c9af89c2e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:51.891627 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c36e38-b258-4de6-bccb-40c9af89c2e2 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:51.891926 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c36e38-b258-4de6-bccb-40c9af89c2e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:51.892213 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c36e38-b258-4de6-bccb-40c9af89c2e2 None admin] PATH_INFO: `/v3/services` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:51.892941 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c36e38-b258-4de6-bccb-40c9af89c2e2 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:51.893255 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c36e38-b258-4de6-bccb-40c9af89c2e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:51.894924 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c36e38-b258-4de6-bccb-40c9af89c2e2 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:51.903671 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 487/958] 10.222.0.99 () {56 vars in 1122 bytes} [Thu Jul 14 15:23:51 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:23:51.907990 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c36e38-b258-4de6-bccb-40c9af89c2e2 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:51.980632 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f77c9f-d1ee-4a0e-aa81-007203510afe None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:52.016967 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:52.025782 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c36e38-b258-4de6-bccb-40c9af89c2e2 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:52.034026 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f35a168-056f-41ab-9094-a27fbc8f37f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:52.034912 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f35a168-056f-41ab-9094-a27fbc8f37f3 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:52.035269 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f35a168-056f-41ab-9094-a27fbc8f37f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:52.035646 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f35a168-056f-41ab-9094-a27fbc8f37f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:52.047725 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f35a168-056f-41ab-9094-a27fbc8f37f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:52.049295 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f35a168-056f-41ab-9094-a27fbc8f37f3 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:52.066121 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e18b30-1893-4d4c-8b01-01b783a04461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:52.067114 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e18b30-1893-4d4c-8b01-01b783a04461 None admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:52.067576 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e18b30-1893-4d4c-8b01-01b783a04461 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:52.068074 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e18b30-1893-4d4c-8b01-01b783a04461 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:52.069005 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e18b30-1893-4d4c-8b01-01b783a04461 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:52.070542 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e18b30-1893-4d4c-8b01-01b783a04461 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:52.074411 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-7f35a168-056f-41ab-9094-a27fbc8f37f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:23:52.074411 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:52.074411 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:52.074411 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:52.074411 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:52.074411 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:52.074411 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:52.074411 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:52.074411 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:52.074411 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:52.074411 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:52.074411 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:52.074411 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:23:52.074411 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:23:52.074411 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:23:52.075313 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:23:52.075313 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:52.075313 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:52.075313 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:52.075313 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:52.075313 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:52.075313 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:52.075313 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:52.075313 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:52.075313 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:52.075313 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:52.075313 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:52.075313 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:52.075313 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:52.075313 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:52.075313 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:52.075313 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:52.076399 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:23:52.076399 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:23:52.076399 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:23:52.076399 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:23:52.083772 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 472/959] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:23:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:52.085364 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f35a168-056f-41ab-9094-a27fbc8f37f3 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:52.098145 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 488/960] 10.222.0.99 () {60 vars in 1144 bytes} [Thu Jul 14 15:23:51 2022] POST /identity/v3/endpoints => generated 349 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 15:23:52.203453 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:52.255798 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f35a168-056f-41ab-9094-a27fbc8f37f3 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:52.332113 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a6dfae-d8c9-48bb-8741-86051980990a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:52.333090 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a6dfae-d8c9-48bb-8741-86051980990a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:52.333553 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a6dfae-d8c9-48bb-8741-86051980990a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:52.334038 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a6dfae-d8c9-48bb-8741-86051980990a None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:52.335236 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a6dfae-d8c9-48bb-8741-86051980990a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:52.335770 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a6dfae-d8c9-48bb-8741-86051980990a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:52.338117 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a6dfae-d8c9-48bb-8741-86051980990a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:52.364588 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 473/961] 10.222.0.99 () {56 vars in 1109 bytes} [Thu Jul 14 15:23:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:52.371376 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e18b30-1893-4d4c-8b01-01b783a04461 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:52.495819 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e18b30-1893-4d4c-8b01-01b783a04461 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:52.540050 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21ca392-ecd2-4d3a-9316-c752a25ebd82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:52.541261 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21ca392-ecd2-4d3a-9316-c752a25ebd82 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:52.541766 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21ca392-ecd2-4d3a-9316-c752a25ebd82 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:52.542245 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21ca392-ecd2-4d3a-9316-c752a25ebd82 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:52.551261 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21ca392-ecd2-4d3a-9316-c752a25ebd82 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:52.553086 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21ca392-ecd2-4d3a-9316-c752a25ebd82 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:52.569178 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-e21ca392-ecd2-4d3a-9316-c752a25ebd82 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 15:23:52.569178 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:52.569178 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:52.569178 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:52.569178 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:52.569178 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:52.569178 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:52.569178 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:52.569178 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:52.569178 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:52.569178 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:52.569178 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:52.569178 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 15:23:52.569178 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 15:23:52.569178 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 15:23:52.569178 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 15:23:52.570171 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:52.570171 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:52.570171 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 15:23:52.570171 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:23:52.570171 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 15:23:52.570171 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:23:52.570171 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:52.570171 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:52.570171 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:52.570171 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:52.570171 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:52.570171 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:52.570171 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:52.570171 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:52.570171 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:52.570171 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:52.570171 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:52.570171 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:52.571097 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:52.571097 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:52.571097 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 15:23:52.571097 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 15:23:52.571097 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 15:23:52.571097 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 15:23:52.571097 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 15:23:52.571097 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 15:23:52.571097 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 15:23:52.571097 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:23:52.573760 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 489/962] 10.222.0.99 () {56 vars in 1117 bytes} [Thu Jul 14 15:23:52 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:52.580944 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a6dfae-d8c9-48bb-8741-86051980990a None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:52.680806 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476c33ff-97bc-457f-a3c4-e6f221c46454 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:52.681780 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476c33ff-97bc-457f-a3c4-e6f221c46454 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:52.682266 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476c33ff-97bc-457f-a3c4-e6f221c46454 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:52.684164 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 490/963] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:23:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 15:23:52.690171 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b44c85-b05d-4cdb-96dc-ccf2c2c6b8cc None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:52.690467 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b44c85-b05d-4cdb-96dc-ccf2c2c6b8cc None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:52.690757 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b44c85-b05d-4cdb-96dc-ccf2c2c6b8cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:52.691153 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a6dfae-d8c9-48bb-8741-86051980990a None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:52.712142 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-a3b44c85-b05d-4cdb-96dc-ccf2c2c6b8cc None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:23:52.713480 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-a3b44c85-b05d-4cdb-96dc-ccf2c2c6b8cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:23:52.716618 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:52.724822 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:52.735730 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0752cf96-5732-4fff-a595-f6e163a63c24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:52.736414 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0752cf96-5732-4fff-a595-f6e163a63c24 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:52.736705 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0752cf96-5732-4fff-a595-f6e163a63c24 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:52.736986 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0752cf96-5732-4fff-a595-f6e163a63c24 None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:52.737835 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0752cf96-5732-4fff-a595-f6e163a63c24 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:52.738149 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0752cf96-5732-4fff-a595-f6e163a63c24 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:52.739587 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0752cf96-5732-4fff-a595-f6e163a63c24 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:52.759467 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 474/964] 10.222.0.99 () {56 vars in 1149 bytes} [Thu Jul 14 15:23:52 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:52.765647 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0752cf96-5732-4fff-a595-f6e163a63c24 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:52.818204 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 491/965] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:23:52 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 15:23:52.840263 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b44c85-b05d-4cdb-96dc-ccf2c2c6b8cc None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:52.886116 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0752cf96-5732-4fff-a595-f6e163a63c24 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:52.922891 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae24e6cb-8cb4-4ec5-a439-3a251634c57d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:52.923554 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae24e6cb-8cb4-4ec5-a439-3a251634c57d None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:52.923880 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae24e6cb-8cb4-4ec5-a439-3a251634c57d None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:52.924160 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae24e6cb-8cb4-4ec5-a439-3a251634c57d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:52.949574 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b44c85-b05d-4cdb-96dc-ccf2c2c6b8cc None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:52.983637 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:52.988970 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22b7bd1-5a7e-48cd-9408-01093f8ae09f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:52.990017 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22b7bd1-5a7e-48cd-9408-01093f8ae09f None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:52.990483 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22b7bd1-5a7e-48cd-9408-01093f8ae09f None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:52.990911 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22b7bd1-5a7e-48cd-9408-01093f8ae09f None admin] PATH_INFO: `/v3/services/compute` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:52.991779 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22b7bd1-5a7e-48cd-9408-01093f8ae09f None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:52.993512 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22b7bd1-5a7e-48cd-9408-01093f8ae09f None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:53.008768 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-b22b7bd1-5a7e-48cd-9408-01093f8ae09f None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 14 15:23:53.008768 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:53.008768 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 15:23:53.008768 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 15:23:53.008768 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 15:23:53.008768 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 15:23:53.008768 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 15:23:53.008768 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 15:23:53.008768 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 14 15:23:53.008768 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:23:53.008768 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 15:23:53.008768 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:23:53.008768 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:53.008768 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:53.008768 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:53.008768 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:53.008768 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:53.010064 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:53.010064 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:53.010064 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:53.010064 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:53.010064 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:53.010064 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:53.010064 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 15:23:53.010064 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 15:23:53.010064 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 15:23:53.010064 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 15:23:53.010064 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:53.010064 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:53.010064 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:53.010064 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:53.010064 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:53.010064 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:53.011066 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:53.011066 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:53.011066 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:53.011066 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:53.011066 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:53.011066 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:53.011066 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:53.011066 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:53.011066 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:53.011066 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:53.011066 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 15:23:53.011066 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 15:23:53.011066 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 14 15:23:53.011066 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:23:53.018212 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae24e6cb-8cb4-4ec5-a439-3a251634c57d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:53.018212 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae24e6cb-8cb4-4ec5-a439-3a251634c57d None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:53.018786 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 492/966] 10.222.0.99 () {56 vars in 1111 bytes} [Thu Jul 14 15:23:52 2022] GET /identity/v3/services/compute => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 15:23:53.019855 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22b7bd1-5a7e-48cd-9408-01093f8ae09f None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:53.024405 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:53.129859 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 475/967] 10.222.0.99 () {58 vars in 1282 bytes} [Thu Jul 14 15:23:52 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 14 15:23:53.131475 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e28a47-39ba-4f87-b3f0-a75ecdad77a3 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:53.131790 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e28a47-39ba-4f87-b3f0-a75ecdad77a3 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:53.132074 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e28a47-39ba-4f87-b3f0-a75ecdad77a3 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:53.135726 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 476/968] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:23:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:23:53.137023 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e28a47-39ba-4f87-b3f0-a75ecdad77a3 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:53.140621 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22b7bd1-5a7e-48cd-9408-01093f8ae09f None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:53.175956 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba32610-87d4-46e5-912e-fe6aa207b9b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:53.176631 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba32610-87d4-46e5-912e-fe6aa207b9b2 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:53.176922 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba32610-87d4-46e5-912e-fe6aa207b9b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:53.177203 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba32610-87d4-46e5-912e-fe6aa207b9b2 None admin] PATH_INFO: `/v3/services` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:53.178101 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba32610-87d4-46e5-912e-fe6aa207b9b2 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:53.178278 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba32610-87d4-46e5-912e-fe6aa207b9b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:53.179671 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba32610-87d4-46e5-912e-fe6aa207b9b2 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:53.184704 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 493/969] 10.222.0.99 () {56 vars in 1112 bytes} [Thu Jul 14 15:23:53 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:53.188999 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f54363-a33a-4638-8562-e1deccc7ade5 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:53.190066 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f54363-a33a-4638-8562-e1deccc7ade5 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:53.190360 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f54363-a33a-4638-8562-e1deccc7ade5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:53.207661 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-87f54363-a33a-4638-8562-e1deccc7ade5 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:23:53.208786 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-87f54363-a33a-4638-8562-e1deccc7ade5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:23:53.234717 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:53.254252 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e28a47-39ba-4f87-b3f0-a75ecdad77a3 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:53.276272 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:53.282898 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 494/970] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:23:53 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 15:23:53.283762 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f54363-a33a-4638-8562-e1deccc7ade5 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:53.310287 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdcad37-6bff-4cfd-99cf-bd4069d5f90c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:53.311270 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdcad37-6bff-4cfd-99cf-bd4069d5f90c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:53.311962 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdcad37-6bff-4cfd-99cf-bd4069d5f90c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:53.312577 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdcad37-6bff-4cfd-99cf-bd4069d5f90c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:53.323790 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdcad37-6bff-4cfd-99cf-bd4069d5f90c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:53.325243 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdcad37-6bff-4cfd-99cf-bd4069d5f90c None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:53.344911 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-efdcad37-6bff-4cfd-99cf-bd4069d5f90c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:23:53.344911 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:53.344911 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:53.344911 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:53.344911 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:53.344911 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:53.344911 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:53.344911 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:53.344911 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:53.344911 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:53.344911 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:53.344911 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:53.344911 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:23:53.344911 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:23:53.344911 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:23:53.345826 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:23:53.345826 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:53.345826 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:53.345826 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:53.345826 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:53.345826 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:53.345826 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:53.345826 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:53.345826 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:53.345826 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:53.345826 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:53.345826 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:53.345826 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:53.345826 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:53.345826 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:53.345826 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:53.345826 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:53.346743 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:23:53.346743 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:23:53.346743 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:23:53.346743 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:23:53.350854 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 477/971] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:23:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:53.351485 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdcad37-6bff-4cfd-99cf-bd4069d5f90c None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:53.391519 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f54363-a33a-4638-8562-e1deccc7ade5 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:53.429929 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036316da-1d64-4848-82c9-66e53de9a419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:53.430628 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036316da-1d64-4848-82c9-66e53de9a419 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:53.430919 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036316da-1d64-4848-82c9-66e53de9a419 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:53.431201 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036316da-1d64-4848-82c9-66e53de9a419 None admin] PATH_INFO: `/v3/services` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:53.431964 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036316da-1d64-4848-82c9-66e53de9a419 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:53.432280 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036316da-1d64-4848-82c9-66e53de9a419 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:53.433680 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036316da-1d64-4848-82c9-66e53de9a419 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:53.447628 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 495/972] 10.222.0.99 () {56 vars in 1112 bytes} [Thu Jul 14 15:23:53 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:53.447628 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036316da-1d64-4848-82c9-66e53de9a419 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:53.474523 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdcad37-6bff-4cfd-99cf-bd4069d5f90c None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:53.513682 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28431d3d-caaf-414c-aa14-640a6be517b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:53.514479 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28431d3d-caaf-414c-aa14-640a6be517b7 None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:53.514951 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28431d3d-caaf-414c-aa14-640a6be517b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:53.515961 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28431d3d-caaf-414c-aa14-640a6be517b7 None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:53.517193 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28431d3d-caaf-414c-aa14-640a6be517b7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:53.518812 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28431d3d-caaf-414c-aa14-640a6be517b7 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:53.525687 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:53.530288 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:53.538908 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.sql.core [None req-28431d3d-caaf-414c-aa14-640a6be517b7 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 14 15:23:53.538908 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 14 15:23:53.538908 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [parameters: {'user_id': '9d577e0b8a0947ada925d2a8d566f007', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 14 15:23:53.538908 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80237) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 14 15:23:53.547889 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-28431d3d-caaf-414c-aa14-640a6be517b7 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 14 15:23:53.547889 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:53.547889 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 14 15:23:53.547889 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 14 15:23:53.547889 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 14 15:23:53.547889 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 14 15:23:53.547889 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 14 15:23:53.547889 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application result = self._query(query) Jul 14 15:23:53.547889 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 14 15:23:53.547889 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application conn.query(q) Jul 14 15:23:53.547889 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 14 15:23:53.547889 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 14 15:23:53.547889 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 14 15:23:53.547889 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application result.read() Jul 14 15:23:53.547889 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 14 15:23:53.548882 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 14 15:23:53.548882 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 14 15:23:53.548882 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 14 15:23:53.548882 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 14 15:23:53.548882 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 14 15:23:53.548882 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 14 15:23:53.548882 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 14 15:23:53.548882 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 14 15:23:53.548882 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:23:53.548882 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 14 15:23:53.548882 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:23:53.548882 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:53.548882 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 14 15:23:53.548882 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 14 15:23:53.548882 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 14 15:23:53.548882 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 14 15:23:53.548882 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 14 15:23:53.548882 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application next(self.gen) Jul 14 15:23:53.549905 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 14 15:23:53.549905 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application yield resource Jul 14 15:23:53.549905 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 14 15:23:53.549905 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application next(self.gen) Jul 14 15:23:53.549905 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 14 15:23:53.549905 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self.session.rollback() Jul 14 15:23:53.549905 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 14 15:23:53.549905 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self.force_reraise() Jul 14 15:23:53.549905 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 14 15:23:53.549905 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise self.value Jul 14 15:23:53.549905 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 14 15:23:53.549905 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 14 15:23:53.549905 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 14 15:23:53.549905 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application session.commit() Jul 14 15:23:53.549905 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 14 15:23:53.549905 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 14 15:23:53.549905 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 14 15:23:53.550801 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._prepare_impl() Jul 14 15:23:53.550801 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 14 15:23:53.550801 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self.session.flush() Jul 14 15:23:53.550801 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 14 15:23:53.550801 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._flush(objects) Jul 14 15:23:53.550801 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 14 15:23:53.550801 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 14 15:23:53.550801 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 14 15:23:53.550801 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application compat.raise_( Jul 14 15:23:53.550801 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 14 15:23:53.550801 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception Jul 14 15:23:53.550801 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 14 15:23:53.550801 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application flush_context.execute() Jul 14 15:23:53.550801 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 14 15:23:53.550801 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rec.execute(self) Jul 14 15:23:53.550801 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 14 15:23:53.550801 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 14 15:23:53.551800 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 14 15:23:53.551800 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 14 15:23:53.551800 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 14 15:23:53.551800 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 14 15:23:53.551800 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 14 15:23:53.551800 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 14 15:23:53.551800 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 14 15:23:53.551800 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 14 15:23:53.551800 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 14 15:23:53.551800 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 14 15:23:53.551800 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 14 15:23:53.551800 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 14 15:23:53.551800 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 14 15:23:53.551800 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 14 15:23:53.551800 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 14 15:23:53.551800 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception Jul 14 15:23:53.552806 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 14 15:23:53.552806 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 14 15:23:53.552806 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 14 15:23:53.552806 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 14 15:23:53.552806 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 14 15:23:53.552806 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application result = self._query(query) Jul 14 15:23:53.552806 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 14 15:23:53.552806 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application conn.query(q) Jul 14 15:23:53.552806 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 14 15:23:53.552806 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 14 15:23:53.552806 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 14 15:23:53.552806 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application result.read() Jul 14 15:23:53.552806 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 14 15:23:53.552806 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 14 15:23:53.552806 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 14 15:23:53.552806 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 14 15:23:53.553820 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 14 15:23:53.553820 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 14 15:23:53.553820 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 14 15:23:53.553820 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 14 15:23:53.553820 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 14 15:23:53.553820 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 14 15:23:53.553820 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [parameters: {'user_id': '9d577e0b8a0947ada925d2a8d566f007', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 14 15:23:53.553820 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 14 15:23:53.553820 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:23:53.553820 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 15:23:53.553820 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:23:53.553820 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:53.553820 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:53.553820 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:53.553820 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:53.554866 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:53.554866 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:53.554866 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:53.554866 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:53.554866 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:53.554866 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:53.554866 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:53.554866 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 14 15:23:53.554866 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 14 15:23:53.554866 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:53.554866 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:53.554866 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 15:23:53.554866 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:23:53.554866 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 15:23:53.554866 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:23:53.554866 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 14 15:23:53.554866 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 14 15:23:53.556429 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 14 15:23:53.556429 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 14 15:23:53.556429 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 14 15:23:53.556429 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 14 15:23:53.556429 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 14 15:23:53.556429 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:23:53.556429 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 478/973] 10.222.0.99 () {60 vars in 1131 bytes} [Thu Jul 14 15:23:53 2022] POST /identity/v3/users => generated 160 bytes in 198 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:53.560198 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28431d3d-caaf-414c-aa14-640a6be517b7 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:53.587790 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036316da-1d64-4848-82c9-66e53de9a419 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:53.631731 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0274e51f-353a-43c4-82a9-19069d7bc681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:53.632466 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0274e51f-353a-43c4-82a9-19069d7bc681 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:53.632766 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0274e51f-353a-43c4-82a9-19069d7bc681 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:53.633057 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0274e51f-353a-43c4-82a9-19069d7bc681 None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:53.633817 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0274e51f-353a-43c4-82a9-19069d7bc681 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:53.634119 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0274e51f-353a-43c4-82a9-19069d7bc681 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:53.635514 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0274e51f-353a-43c4-82a9-19069d7bc681 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:53.652341 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 496/974] 10.222.0.99 () {56 vars in 1109 bytes} [Thu Jul 14 15:23:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:53.660475 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0274e51f-353a-43c4-82a9-19069d7bc681 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:53.686050 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28431d3d-caaf-414c-aa14-640a6be517b7 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:53.724296 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e21bbf4-a475-4210-a76f-04fbfbf5c741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:53.725256 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e21bbf4-a475-4210-a76f-04fbfbf5c741 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:53.725500 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e21bbf4-a475-4210-a76f-04fbfbf5c741 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:53.725902 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e21bbf4-a475-4210-a76f-04fbfbf5c741 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:53.734683 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e21bbf4-a475-4210-a76f-04fbfbf5c741 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:53.736478 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e21bbf4-a475-4210-a76f-04fbfbf5c741 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:53.752741 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-4e21bbf4-a475-4210-a76f-04fbfbf5c741 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 14 15:23:53.752741 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:53.752741 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:53.752741 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:53.752741 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:53.752741 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:53.752741 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:53.752741 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:53.752741 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:53.752741 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:53.752741 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:53.752741 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:53.752741 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 15:23:53.752741 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 15:23:53.752741 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 15:23:53.752741 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 15:23:53.753717 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:53.753717 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:53.753717 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 15:23:53.753717 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:23:53.753717 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 15:23:53.753717 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:23:53.753717 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:53.753717 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:53.753717 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:53.753717 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:53.753717 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:53.753717 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:53.753717 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:53.753717 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:53.753717 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:53.753717 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:53.753717 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:53.753717 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:53.754656 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:53.754656 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:53.754656 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 15:23:53.754656 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 15:23:53.754656 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 15:23:53.754656 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 15:23:53.754656 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 15:23:53.754656 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 15:23:53.754656 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 14 15:23:53.754656 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:23:53.758592 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 479/975] 10.222.0.99 () {56 vars in 1099 bytes} [Thu Jul 14 15:23:53 2022] GET /identity/v3/users/cinder => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 15:23:53.760007 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e21bbf4-a475-4210-a76f-04fbfbf5c741 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:53.776965 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0274e51f-353a-43c4-82a9-19069d7bc681 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:53.801416 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:53.814798 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4676b6d-03be-4443-8116-f94bcf6c3ac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:53.815981 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4676b6d-03be-4443-8116-f94bcf6c3ac9 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:53.816503 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4676b6d-03be-4443-8116-f94bcf6c3ac9 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:53.816988 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4676b6d-03be-4443-8116-f94bcf6c3ac9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:53.824409 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4676b6d-03be-4443-8116-f94bcf6c3ac9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:53.826178 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4676b6d-03be-4443-8116-f94bcf6c3ac9 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:53.826974 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:53.843383 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-a4676b6d-03be-4443-8116-f94bcf6c3ac9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:23:53.843383 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:53.843383 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:53.843383 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:53.843383 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:53.843383 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:53.843383 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:53.843383 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:53.843383 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:53.843383 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:53.843383 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:53.843383 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:53.843383 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 15:23:53.843383 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 15:23:53.843383 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 15:23:53.844524 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 15:23:53.844524 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:53.844524 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:53.844524 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:53.844524 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:53.844524 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:53.844524 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:53.844524 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:53.844524 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:53.844524 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:53.844524 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:53.844524 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:53.844524 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:53.844524 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:53.844524 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:53.844524 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:53.844524 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:53.845400 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 15:23:53.845400 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 15:23:53.845400 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 15:23:53.845400 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 15:23:53.845400 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 15:23:53.845400 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 15:23:53.845400 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:23:53.845400 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:23:53.847988 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 497/976] 10.222.0.99 () {56 vars in 1111 bytes} [Thu Jul 14 15:23:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 15:23:53.856575 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4676b6d-03be-4443-8116-f94bcf6c3ac9 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:53.894662 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e21bbf4-a475-4210-a76f-04fbfbf5c741 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:53.944026 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a1afe3-3846-44a7-9973-2325ab4a45e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:53.944710 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a1afe3-3846-44a7-9973-2325ab4a45e8 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:53.945007 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a1afe3-3846-44a7-9973-2325ab4a45e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:53.945290 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a1afe3-3846-44a7-9973-2325ab4a45e8 None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:53.946386 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a1afe3-3846-44a7-9973-2325ab4a45e8 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:53.946702 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a1afe3-3846-44a7-9973-2325ab4a45e8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:53.948526 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a1afe3-3846-44a7-9973-2325ab4a45e8 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:53.989426 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4676b6d-03be-4443-8116-f94bcf6c3ac9 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:54.011736 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 480/977] 10.222.0.99 () {56 vars in 1101 bytes} [Thu Jul 14 15:23:53 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:23:54.045568 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8b37c1-7636-4395-8d62-70788d62652d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:54.046245 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8b37c1-7636-4395-8d62-70788d62652d None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:54.046528 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8b37c1-7636-4395-8d62-70788d62652d None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:54.046879 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8b37c1-7636-4395-8d62-70788d62652d None admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:54.047791 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8b37c1-7636-4395-8d62-70788d62652d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:54.048191 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8b37c1-7636-4395-8d62-70788d62652d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:54.049680 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8b37c1-7636-4395-8d62-70788d62652d None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:54.067803 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 498/978] 10.222.0.99 () {56 vars in 1148 bytes} [Thu Jul 14 15:23:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:54.069968 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a1afe3-3846-44a7-9973-2325ab4a45e8 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:54.182570 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a1afe3-3846-44a7-9973-2325ab4a45e8 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:54.200407 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:54.222188 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81874ae-cc46-45c2-bb03-68f15de185e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:54.222858 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81874ae-cc46-45c2-bb03-68f15de185e1 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:54.223144 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81874ae-cc46-45c2-bb03-68f15de185e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:54.223423 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81874ae-cc46-45c2-bb03-68f15de185e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:54.224295 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81874ae-cc46-45c2-bb03-68f15de185e1 None admin] RBAC: Adding query filter params (role.id=ebca1dcd25d64cebaae5f4c18285be4a, scope.project.id=9048d1a6f00443ec92f11471babe5a56, user.id=adaeadf613954d7393b3fd3bae68c362) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:54.224606 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81874ae-cc46-45c2-bb03-68f15de185e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:54.226037 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81874ae-cc46-45c2-bb03-68f15de185e1 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:54.232872 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 481/979] 10.222.0.99 () {56 vars in 1374 bytes} [Thu Jul 14 15:23:54 2022] GET /identity/v3/role_assignments?user.id=adaeadf613954d7393b3fd3bae68c362&scope.project.id=9048d1a6f00443ec92f11471babe5a56&role.id=ebca1dcd25d64cebaae5f4c18285be4a => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:54.416731 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0ce4bf-ee61-4eac-a324-c05d7863c694 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:54.417120 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0ce4bf-ee61-4eac-a324-c05d7863c694 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:54.417296 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0ce4bf-ee61-4eac-a324-c05d7863c694 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:54.418711 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 499/980] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:23:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:23:54.424366 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293a1945-5729-47d8-add8-102cca024168 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:54.424662 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293a1945-5729-47d8-add8-102cca024168 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:54.424942 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293a1945-5729-47d8-add8-102cca024168 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:54.442983 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-293a1945-5729-47d8-add8-102cca024168 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:23:54.444530 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-293a1945-5729-47d8-add8-102cca024168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:23:54.521135 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 482/981] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:23:54 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 15:23:54.543373 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0ce4bf-ee61-4eac-a324-c05d7863c694 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:54.589787 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:54.651668 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0ce4bf-ee61-4eac-a324-c05d7863c694 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:54.690355 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77cdfa3-cc28-4010-9d88-8cbaf4ab534b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:54.691072 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77cdfa3-cc28-4010-9d88-8cbaf4ab534b None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:54.691359 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77cdfa3-cc28-4010-9d88-8cbaf4ab534b None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:54.691717 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77cdfa3-cc28-4010-9d88-8cbaf4ab534b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:54.692391 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77cdfa3-cc28-4010-9d88-8cbaf4ab534b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:54.693770 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77cdfa3-cc28-4010-9d88-8cbaf4ab534b None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:54.706047 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-c77cdfa3-cc28-4010-9d88-8cbaf4ab534b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 15:23:54.706047 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:54.706047 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 15:23:54.706047 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 15:23:54.706047 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 15:23:54.706047 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 15:23:54.706047 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 15:23:54.706047 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 15:23:54.706047 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 15:23:54.706047 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:23:54.706047 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 15:23:54.706047 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:23:54.706047 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:54.706047 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:54.706047 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:54.706047 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:54.706047 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:54.707024 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:54.707024 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:54.707024 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:54.707024 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:54.707024 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:54.707024 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:54.707024 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 15:23:54.707024 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 15:23:54.707024 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 15:23:54.707024 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 15:23:54.707024 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:54.707024 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:54.707024 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:54.707024 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:54.707024 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:54.707024 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:54.708137 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:54.708137 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:54.708137 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:54.708137 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:54.708137 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:54.708137 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:54.708137 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:54.708137 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:54.708137 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:54.708137 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:54.708137 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 15:23:54.708137 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 15:23:54.708137 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 15:23:54.708137 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:23:54.708137 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 500/982] 10.222.0.99 () {56 vars in 1126 bytes} [Thu Jul 14 15:23:54 2022] GET /identity/v3/services/object-store => generated 93 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:54.715161 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293a1945-5729-47d8-add8-102cca024168 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:54.818824 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293a1945-5729-47d8-add8-102cca024168 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:54.829933 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:54.862570 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f79b3d0-08a1-4525-ba9a-2dcb18d01552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:54.863252 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f79b3d0-08a1-4525-ba9a-2dcb18d01552 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:54.863543 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f79b3d0-08a1-4525-ba9a-2dcb18d01552 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:54.863953 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f79b3d0-08a1-4525-ba9a-2dcb18d01552 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:54.865347 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f79b3d0-08a1-4525-ba9a-2dcb18d01552 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:54.865928 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f79b3d0-08a1-4525-ba9a-2dcb18d01552 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:54.868129 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f79b3d0-08a1-4525-ba9a-2dcb18d01552 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:54.879268 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 483/983] 10.222.0.99 () {56 vars in 1122 bytes} [Thu Jul 14 15:23:54 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:23:54.880356 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77cdfa3-cc28-4010-9d88-8cbaf4ab534b None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:54.992398 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77cdfa3-cc28-4010-9d88-8cbaf4ab534b None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:55.001974 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:55.030745 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49f1ff1-369a-402d-957b-590648fa3acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:55.031708 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49f1ff1-369a-402d-957b-590648fa3acf None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:55.031982 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49f1ff1-369a-402d-957b-590648fa3acf None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:55.032278 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49f1ff1-369a-402d-957b-590648fa3acf None admin] PATH_INFO: `/v3/services` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:55.032994 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49f1ff1-369a-402d-957b-590648fa3acf None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:55.033313 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49f1ff1-369a-402d-957b-590648fa3acf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:55.034701 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49f1ff1-369a-402d-957b-590648fa3acf None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:55.044973 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 501/984] 10.222.0.99 () {56 vars in 1122 bytes} [Thu Jul 14 15:23:54 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:23:55.045925 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f79b3d0-08a1-4525-ba9a-2dcb18d01552 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:55.153592 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f79b3d0-08a1-4525-ba9a-2dcb18d01552 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:55.189680 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1770ec-1c20-4917-b3a2-a38d3a5296a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:55.190347 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1770ec-1c20-4917-b3a2-a38d3a5296a3 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:55.190638 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1770ec-1c20-4917-b3a2-a38d3a5296a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:55.190921 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1770ec-1c20-4917-b3a2-a38d3a5296a3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:55.191753 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1770ec-1c20-4917-b3a2-a38d3a5296a3 None admin] RBAC: Adding query filter params (interface=admin, service_id=079d217a520c42e2b41b996c39af0794, region_id=RegionOne) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:55.192075 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1770ec-1c20-4917-b3a2-a38d3a5296a3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:55.193458 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1770ec-1c20-4917-b3a2-a38d3a5296a3 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:55.206510 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 484/985] 10.222.0.99 () {56 vars in 1249 bytes} [Thu Jul 14 15:23:55 2022] GET /identity/v3/endpoints?service_id=079d217a520c42e2b41b996c39af0794&interface=admin®ion_id=RegionOne => generated 193 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:23:55.207775 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49f1ff1-369a-402d-957b-590648fa3acf None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:55.326694 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49f1ff1-369a-402d-957b-590648fa3acf None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:55.361436 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35474841-10ea-46f7-b3f5-c7bf73c06686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:55.362111 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35474841-10ea-46f7-b3f5-c7bf73c06686 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:55.362401 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35474841-10ea-46f7-b3f5-c7bf73c06686 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:55.362681 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35474841-10ea-46f7-b3f5-c7bf73c06686 None admin] PATH_INFO: `/v3/services` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:55.363369 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35474841-10ea-46f7-b3f5-c7bf73c06686 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:55.364788 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35474841-10ea-46f7-b3f5-c7bf73c06686 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:55.379900 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 502/986] 10.222.0.99 () {56 vars in 1087 bytes} [Thu Jul 14 15:23:55 2022] GET /identity/v3/services => generated 1497 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 15:23:55.629020 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293bced1-fcfa-4955-b256-ff2716cf765a None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:55.629360 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293bced1-fcfa-4955-b256-ff2716cf765a None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:55.629553 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293bced1-fcfa-4955-b256-ff2716cf765a None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:55.631013 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 485/987] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:23:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:23:55.636154 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073306da-780b-496e-8c49-ff772daa7e1d None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:55.636449 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073306da-780b-496e-8c49-ff772daa7e1d None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:55.636985 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073306da-780b-496e-8c49-ff772daa7e1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:55.653940 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-073306da-780b-496e-8c49-ff772daa7e1d None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:23:55.655117 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-073306da-780b-496e-8c49-ff772daa7e1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:23:55.682948 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:55.725188 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 503/988] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:23:55 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 15:23:55.748117 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293bced1-fcfa-4955-b256-ff2716cf765a None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:55.790599 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:55.854488 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293bced1-fcfa-4955-b256-ff2716cf765a None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:55.889964 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1921abf-0637-41db-9c23-c17c2be2e0d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:55.890687 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1921abf-0637-41db-9c23-c17c2be2e0d6 None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:55.890985 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1921abf-0637-41db-9c23-c17c2be2e0d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:55.891271 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1921abf-0637-41db-9c23-c17c2be2e0d6 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:55.892031 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1921abf-0637-41db-9c23-c17c2be2e0d6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:55.893408 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1921abf-0637-41db-9c23-c17c2be2e0d6 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:55.909816 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 486/989] 10.222.0.99 () {60 vars in 1141 bytes} [Thu Jul 14 15:23:55 2022] POST /identity/v3/services => generated 242 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:23:56.420522 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3db4867-661a-47aa-bd45-1f0b8f3732d1 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:56.421138 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3db4867-661a-47aa-bd45-1f0b8f3732d1 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:56.421138 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3db4867-661a-47aa-bd45-1f0b8f3732d1 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:56.424241 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 504/990] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:23:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 15:23:56.429650 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bd6bbc-de3d-4c88-90db-66c5fe0d975b None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:56.430157 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bd6bbc-de3d-4c88-90db-66c5fe0d975b None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:56.430237 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bd6bbc-de3d-4c88-90db-66c5fe0d975b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:56.449731 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-b2bd6bbc-de3d-4c88-90db-66c5fe0d975b None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:23:56.450972 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b2bd6bbc-de3d-4c88-90db-66c5fe0d975b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:23:56.539275 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 487/991] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:23:56 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 15:23:56.565372 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3db4867-661a-47aa-bd45-1f0b8f3732d1 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:56.688897 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3db4867-661a-47aa-bd45-1f0b8f3732d1 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:56.732524 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9476924-4a97-421d-9d1e-51d5c93e5054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:56.733330 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9476924-4a97-421d-9d1e-51d5c93e5054 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:56.733617 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9476924-4a97-421d-9d1e-51d5c93e5054 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:56.733896 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9476924-4a97-421d-9d1e-51d5c93e5054 None admin] PATH_INFO: `/v3/services/image` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:56.735207 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9476924-4a97-421d-9d1e-51d5c93e5054 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:56.738703 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9476924-4a97-421d-9d1e-51d5c93e5054 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:56.750626 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-b9476924-4a97-421d-9d1e-51d5c93e5054 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 14 15:23:56.750626 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:56.750626 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 15:23:56.750626 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 15:23:56.750626 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 15:23:56.750626 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 15:23:56.750626 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 15:23:56.750626 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 15:23:56.750626 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 14 15:23:56.750626 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:23:56.750626 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 15:23:56.750626 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:23:56.750626 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:56.750626 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:56.750626 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:56.750626 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:56.750626 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:56.752852 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:56.752852 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:56.752852 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:56.752852 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:56.752852 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:56.752852 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:56.752852 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 15:23:56.752852 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 15:23:56.752852 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 15:23:56.752852 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 15:23:56.752852 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:56.752852 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:56.752852 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:56.752852 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:56.752852 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:56.752852 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:56.753699 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:56.753699 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:56.753699 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:56.753699 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:56.753699 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:56.753699 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:56.753699 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:56.753699 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:56.753699 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:56.753699 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:56.753699 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 15:23:56.753699 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 15:23:56.753699 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 14 15:23:56.753699 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:23:56.755932 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 505/992] 10.222.0.99 () {56 vars in 1105 bytes} [Thu Jul 14 15:23:56 2022] GET /identity/v3/services/image => generated 86 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:56.757746 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bd6bbc-de3d-4c88-90db-66c5fe0d975b None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:56.784717 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:56.885372 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bd6bbc-de3d-4c88-90db-66c5fe0d975b None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:56.926713 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7fd518-6f2d-4c7d-9c4e-627d9f0e48ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:56.926713 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7fd518-6f2d-4c7d-9c4e-627d9f0e48ad None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:56.927025 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7fd518-6f2d-4c7d-9c4e-627d9f0e48ad None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:56.927279 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7fd518-6f2d-4c7d-9c4e-627d9f0e48ad None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:56.928163 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7fd518-6f2d-4c7d-9c4e-627d9f0e48ad None admin] RBAC: Adding query filter params (name=image) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:56.929348 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7fd518-6f2d-4c7d-9c4e-627d9f0e48ad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:56.929849 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7fd518-6f2d-4c7d-9c4e-627d9f0e48ad None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:56.945424 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 488/993] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:23:56 2022] GET /identity/v3/services?name=image => generated 122 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:56.946495 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9476924-4a97-421d-9d1e-51d5c93e5054 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:56.994987 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:57.088226 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9476924-4a97-421d-9d1e-51d5c93e5054 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:57.131381 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cbc2bf-6752-4b1b-8fdb-cf909ae81dfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:57.132097 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cbc2bf-6752-4b1b-8fdb-cf909ae81dfd None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:57.132389 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cbc2bf-6752-4b1b-8fdb-cf909ae81dfd None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:57.132672 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cbc2bf-6752-4b1b-8fdb-cf909ae81dfd None admin] PATH_INFO: `/v3/services` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:57.133400 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cbc2bf-6752-4b1b-8fdb-cf909ae81dfd None admin] RBAC: Adding query filter params (type=image) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:57.133711 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cbc2bf-6752-4b1b-8fdb-cf909ae81dfd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:57.135111 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cbc2bf-6752-4b1b-8fdb-cf909ae81dfd None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:57.141007 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 506/994] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:23:56 2022] GET /identity/v3/services?type=image => generated 122 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:57.519399 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524bd415-bca1-441a-82c9-784861eb6039 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:57.519961 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524bd415-bca1-441a-82c9-784861eb6039 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:57.519961 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524bd415-bca1-441a-82c9-784861eb6039 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:57.521612 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 489/995] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:23:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:23:57.527082 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f03658f-67d6-45e8-8ab7-006254d46cbc None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:57.527516 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f03658f-67d6-45e8-8ab7-006254d46cbc None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:57.527734 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f03658f-67d6-45e8-8ab7-006254d46cbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:57.548336 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-6f03658f-67d6-45e8-8ab7-006254d46cbc None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:23:57.549620 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-6f03658f-67d6-45e8-8ab7-006254d46cbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:23:57.635991 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 507/996] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:23:57 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 15:23:57.661289 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524bd415-bca1-441a-82c9-784861eb6039 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:57.774034 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:57.792766 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524bd415-bca1-441a-82c9-784861eb6039 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:57.834054 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae6d598-384b-4b6e-89ab-8af8d27f1484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:57.834874 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae6d598-384b-4b6e-89ab-8af8d27f1484 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:57.835157 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae6d598-384b-4b6e-89ab-8af8d27f1484 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:57.835434 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae6d598-384b-4b6e-89ab-8af8d27f1484 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:57.836195 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae6d598-384b-4b6e-89ab-8af8d27f1484 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:57.837507 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae6d598-384b-4b6e-89ab-8af8d27f1484 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:57.850667 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-0ae6d598-384b-4b6e-89ab-8af8d27f1484 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 15:23:57.850667 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:57.850667 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 15:23:57.850667 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 15:23:57.850667 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 15:23:57.850667 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 15:23:57.850667 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 15:23:57.850667 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 15:23:57.850667 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 15:23:57.850667 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:23:57.850667 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 15:23:57.850667 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:23:57.850667 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:57.850667 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:57.850667 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:57.850667 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:57.850667 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:57.852218 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:57.852218 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:57.852218 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:57.852218 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:57.852218 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:57.852218 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:57.852218 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 15:23:57.852218 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 15:23:57.852218 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 15:23:57.852218 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 15:23:57.852218 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:57.852218 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:57.852218 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:57.852218 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:57.852218 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:57.852218 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:57.853201 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:57.853201 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:57.853201 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:57.853201 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:57.853201 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:57.853201 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:57.853201 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:57.853201 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:57.853201 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:57.853201 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:57.853201 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 15:23:57.853201 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 15:23:57.853201 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 15:23:57.853201 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:23:57.856544 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 490/997] 10.222.0.99 () {56 vars in 1126 bytes} [Thu Jul 14 15:23:57 2022] GET /identity/v3/services/object-store => generated 93 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:23:57.858107 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f03658f-67d6-45e8-8ab7-006254d46cbc None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:57.893711 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:57.989534 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f03658f-67d6-45e8-8ab7-006254d46cbc None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:58.036904 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fa452e-43a2-4aa5-bcb6-fa0840bdc41c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:58.036904 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fa452e-43a2-4aa5-bcb6-fa0840bdc41c None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:58.037838 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fa452e-43a2-4aa5-bcb6-fa0840bdc41c None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:58.037838 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fa452e-43a2-4aa5-bcb6-fa0840bdc41c None admin] PATH_INFO: `/v3/services` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:58.038163 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fa452e-43a2-4aa5-bcb6-fa0840bdc41c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:58.038399 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fa452e-43a2-4aa5-bcb6-fa0840bdc41c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:58.039813 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fa452e-43a2-4aa5-bcb6-fa0840bdc41c None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:58.054296 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 508/998] 10.222.0.99 () {56 vars in 1122 bytes} [Thu Jul 14 15:23:57 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:58.055228 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae6d598-384b-4b6e-89ab-8af8d27f1484 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:58.059792 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e7fae0-eefc-40bf-90e3-e07f3cd34625 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:58.060086 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e7fae0-eefc-40bf-90e3-e07f3cd34625 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:58.060369 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e7fae0-eefc-40bf-90e3-e07f3cd34625 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:58.061856 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 509/999] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:23:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 15:23:58.068065 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a7ae63-0ea1-4f64-b571-f37bded8ac14 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:58.068361 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a7ae63-0ea1-4f64-b571-f37bded8ac14 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:58.068644 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a7ae63-0ea1-4f64-b571-f37bded8ac14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:58.086383 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-a8a7ae63-0ea1-4f64-b571-f37bded8ac14 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:23:58.087588 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-a8a7ae63-0ea1-4f64-b571-f37bded8ac14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:23:58.164560 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 510/1000] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:23:58 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 15:23:58.172704 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae6d598-384b-4b6e-89ab-8af8d27f1484 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:58.187357 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a7ae63-0ea1-4f64-b571-f37bded8ac14 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:58.204829 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:58.208661 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f956972-3667-47a1-b52d-5df949e6c6a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:58.209977 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f956972-3667-47a1-b52d-5df949e6c6a6 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:58.210447 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f956972-3667-47a1-b52d-5df949e6c6a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:58.210904 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f956972-3667-47a1-b52d-5df949e6c6a6 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:58.211812 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f956972-3667-47a1-b52d-5df949e6c6a6 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:58.212326 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f956972-3667-47a1-b52d-5df949e6c6a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:58.213900 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f956972-3667-47a1-b52d-5df949e6c6a6 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:58.224310 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 491/1001] 10.222.0.99 () {56 vars in 1122 bytes} [Thu Jul 14 15:23:58 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:23:58.225367 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f956972-3667-47a1-b52d-5df949e6c6a6 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:58.248742 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:58.295045 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a7ae63-0ea1-4f64-b571-f37bded8ac14 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:58.333615 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ee85cd-6139-4a1b-b090-9ffc861ae11a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:58.334424 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ee85cd-6139-4a1b-b090-9ffc861ae11a None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:58.334705 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ee85cd-6139-4a1b-b090-9ffc861ae11a None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:58.334981 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ee85cd-6139-4a1b-b090-9ffc861ae11a None admin] PATH_INFO: `/v3/services/compute` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:58.335713 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ee85cd-6139-4a1b-b090-9ffc861ae11a None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:58.337169 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ee85cd-6139-4a1b-b090-9ffc861ae11a None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:58.341093 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f956972-3667-47a1-b52d-5df949e6c6a6 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:58.349693 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-a7ee85cd-6139-4a1b-b090-9ffc861ae11a None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 14 15:23:58.349693 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:58.349693 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 15:23:58.349693 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 15:23:58.349693 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 15:23:58.349693 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 15:23:58.349693 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 15:23:58.349693 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 15:23:58.349693 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 14 15:23:58.349693 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:23:58.349693 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 15:23:58.349693 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:23:58.349693 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:23:58.349693 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:23:58.349693 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:23:58.349693 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:23:58.349693 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:23:58.350695 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:23:58.350695 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:23:58.350695 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:23:58.350695 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:23:58.350695 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:23:58.350695 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:23:58.350695 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 15:23:58.350695 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 15:23:58.350695 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 15:23:58.350695 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 15:23:58.350695 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:23:58.350695 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:23:58.350695 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:23:58.350695 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:23:58.350695 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:23:58.350695 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:23:58.351525 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:23:58.351525 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:23:58.351525 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:23:58.351525 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:23:58.351525 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:23:58.351525 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:23:58.351525 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:23:58.351525 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:23:58.351525 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:23:58.351525 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:23:58.351525 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 15:23:58.351525 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 15:23:58.351525 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 14 15:23:58.351525 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:23:58.355754 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 511/1002] 10.222.0.99 () {56 vars in 1111 bytes} [Thu Jul 14 15:23:58 2022] GET /identity/v3/services/compute => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 15:23:58.356968 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ee85cd-6139-4a1b-b090-9ffc861ae11a None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:58.383314 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd174c5-e3ce-4be0-b620-5021155278cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:58.384128 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd174c5-e3ce-4be0-b620-5021155278cd None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:58.384423 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd174c5-e3ce-4be0-b620-5021155278cd None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:58.384704 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd174c5-e3ce-4be0-b620-5021155278cd None admin] PATH_INFO: `/v3/endpoints` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:58.385455 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd174c5-e3ce-4be0-b620-5021155278cd None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:58.387166 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd174c5-e3ce-4be0-b620-5021155278cd None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:58.411631 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 492/1003] 10.222.0.99 () {60 vars in 1144 bytes} [Thu Jul 14 15:23:58 2022] POST /identity/v3/endpoints => generated 325 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:23:58.463336 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:58.487412 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ee85cd-6139-4a1b-b090-9ffc861ae11a None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:58.535954 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c525a2-3b4d-4d1d-9355-4b7491df3db2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:58.536672 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c525a2-3b4d-4d1d-9355-4b7491df3db2 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:58.536916 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c525a2-3b4d-4d1d-9355-4b7491df3db2 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:58.537199 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c525a2-3b4d-4d1d-9355-4b7491df3db2 None admin] PATH_INFO: `/v3/services` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:58.537924 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c525a2-3b4d-4d1d-9355-4b7491df3db2 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:58.538237 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c525a2-3b4d-4d1d-9355-4b7491df3db2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:58.539649 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c525a2-3b4d-4d1d-9355-4b7491df3db2 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:58.549669 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 512/1004] 10.222.0.99 () {56 vars in 1112 bytes} [Thu Jul 14 15:23:58 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:23:58.552478 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd174c5-e3ce-4be0-b620-5021155278cd None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:58.639534 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:58.664258 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd174c5-e3ce-4be0-b620-5021155278cd None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:58.699855 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9767dd0d-18a8-40f7-989b-b9a15f689ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:58.700525 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9767dd0d-18a8-40f7-989b-b9a15f689ad2 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:58.700847 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9767dd0d-18a8-40f7-989b-b9a15f689ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:58.701132 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9767dd0d-18a8-40f7-989b-b9a15f689ad2 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:58.701944 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9767dd0d-18a8-40f7-989b-b9a15f689ad2 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:58.702310 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9767dd0d-18a8-40f7-989b-b9a15f689ad2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:58.704277 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9767dd0d-18a8-40f7-989b-b9a15f689ad2 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:58.709693 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 493/1005] 10.222.0.99 () {56 vars in 1112 bytes} [Thu Jul 14 15:23:58 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:23:58.715320 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c525a2-3b4d-4d1d-9355-4b7491df3db2 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:58.822751 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c525a2-3b4d-4d1d-9355-4b7491df3db2 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:58.857514 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d761e168-bd2a-44ee-84e0-f6c037cbc63f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:58.858177 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d761e168-bd2a-44ee-84e0-f6c037cbc63f None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:58.858466 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d761e168-bd2a-44ee-84e0-f6c037cbc63f None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:58.858739 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d761e168-bd2a-44ee-84e0-f6c037cbc63f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:58.859846 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d761e168-bd2a-44ee-84e0-f6c037cbc63f None admin] RBAC: Adding query filter params (interface=public, service_id=8bbf6213287743a98df6863b943063ea, region_id=RegionOne) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:23:58.860226 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d761e168-bd2a-44ee-84e0-f6c037cbc63f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:58.861695 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d761e168-bd2a-44ee-84e0-f6c037cbc63f None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:58.871461 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 513/1006] 10.222.0.99 () {56 vars in 1251 bytes} [Thu Jul 14 15:23:58 2022] GET /identity/v3/endpoints?service_id=8bbf6213287743a98df6863b943063ea&interface=public®ion_id=RegionOne => generated 194 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:23:58.872982 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9767dd0d-18a8-40f7-989b-b9a15f689ad2 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:58.982166 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9767dd0d-18a8-40f7-989b-b9a15f689ad2 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:59.023248 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d96f4f-dc38-4c4b-972e-d03d6ce726c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:59.024029 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d96f4f-dc38-4c4b-972e-d03d6ce726c2 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:59.024320 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d96f4f-dc38-4c4b-972e-d03d6ce726c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:59.024600 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d96f4f-dc38-4c4b-972e-d03d6ce726c2 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:59.025294 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d96f4f-dc38-4c4b-972e-d03d6ce726c2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:59.026687 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d96f4f-dc38-4c4b-972e-d03d6ce726c2 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:59.035046 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 494/1007] 10.222.0.99 () {56 vars in 1087 bytes} [Thu Jul 14 15:23:58 2022] GET /identity/v3/services => generated 1727 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 15:23:59.272676 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016e8d1d-fd1d-44af-b065-9309812f2d20 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:59.272971 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016e8d1d-fd1d-44af-b065-9309812f2d20 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:59.273260 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016e8d1d-fd1d-44af-b065-9309812f2d20 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:59.274640 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 514/1008] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:23:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:23:59.280015 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d101e51-4a8a-4fec-a96c-931822862b86 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:59.280366 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d101e51-4a8a-4fec-a96c-931822862b86 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:59.280650 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d101e51-4a8a-4fec-a96c-931822862b86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:59.287475 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:59.298929 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-5d101e51-4a8a-4fec-a96c-931822862b86 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:23:59.300171 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-5d101e51-4a8a-4fec-a96c-931822862b86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:23:59.369138 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 495/1009] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:23:59 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 15:23:59.396648 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016e8d1d-fd1d-44af-b065-9309812f2d20 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:23:59.402858 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:23:59.507722 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016e8d1d-fd1d-44af-b065-9309812f2d20 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:23:59.551019 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a027b78d-98c7-432c-a089-59c5bbd75adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:23:59.551643 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a027b78d-98c7-432c-a089-59c5bbd75adf None admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:23:59.551913 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a027b78d-98c7-432c-a089-59c5bbd75adf None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:23:59.552192 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a027b78d-98c7-432c-a089-59c5bbd75adf None admin] PATH_INFO: `/v3/services` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:23:59.552941 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a027b78d-98c7-432c-a089-59c5bbd75adf None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:23:59.554308 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a027b78d-98c7-432c-a089-59c5bbd75adf None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:23:59.569263 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 515/1010] 10.222.0.99 () {60 vars in 1141 bytes} [Thu Jul 14 15:23:59 2022] POST /identity/v3/services => generated 242 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:24:00.548643 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88248cbc-f38f-4c53-babb-8c76c560c615 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:00.549165 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88248cbc-f38f-4c53-babb-8c76c560c615 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:00.549262 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88248cbc-f38f-4c53-babb-8c76c560c615 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:00.550920 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 496/1011] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:24:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:24:00.556086 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a492c7-ead7-4f04-8056-f12092d83939 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:00.556489 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a492c7-ead7-4f04-8056-f12092d83939 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:00.556657 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a492c7-ead7-4f04-8056-f12092d83939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:00.574546 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-32a492c7-ead7-4f04-8056-f12092d83939 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:24:00.575798 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-32a492c7-ead7-4f04-8056-f12092d83939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:24:00.638289 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:24:00.656446 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 516/1012] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:24:00 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 15:24:00.680537 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88248cbc-f38f-4c53-babb-8c76c560c615 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:00.786318 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88248cbc-f38f-4c53-babb-8c76c560c615 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:00.820787 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4e16be-1f5e-484f-aa7e-84cabf030acb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:00.821496 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4e16be-1f5e-484f-aa7e-84cabf030acb None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:00.821782 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4e16be-1f5e-484f-aa7e-84cabf030acb None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:00.822060 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4e16be-1f5e-484f-aa7e-84cabf030acb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:00.823225 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4e16be-1f5e-484f-aa7e-84cabf030acb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:00.824681 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4e16be-1f5e-484f-aa7e-84cabf030acb None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:00.826504 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 497/1013] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:24:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:24:00.832823 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a492c7-ead7-4f04-8056-f12092d83939 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:00.939120 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a492c7-ead7-4f04-8056-f12092d83939 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:00.974478 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603445cd-f7fa-44f4-a77b-6e725a065fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:00.975582 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603445cd-f7fa-44f4-a77b-6e725a065fff None admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:00.975582 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603445cd-f7fa-44f4-a77b-6e725a065fff None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:00.975912 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603445cd-f7fa-44f4-a77b-6e725a065fff None admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:00.976620 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603445cd-f7fa-44f4-a77b-6e725a065fff None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:00.978055 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603445cd-f7fa-44f4-a77b-6e725a065fff None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:01.007135 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 517/1014] 10.222.0.99 () {60 vars in 1141 bytes} [Thu Jul 14 15:24:00 2022] POST /identity/v3/projects => generated 311 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:24:01.121825 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c837c5-d83b-45ec-a4da-caabb7d73081 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:01.122049 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c837c5-d83b-45ec-a4da-caabb7d73081 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:01.122316 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c837c5-d83b-45ec-a4da-caabb7d73081 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:01.123764 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 498/1015] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:24:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:24:01.130707 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d893a11-9c11-4231-a5e6-2169c66b494e None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:01.131000 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d893a11-9c11-4231-a5e6-2169c66b494e None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:01.131850 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d893a11-9c11-4231-a5e6-2169c66b494e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:01.151051 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-3d893a11-9c11-4231-a5e6-2169c66b494e None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:24:01.152299 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-3d893a11-9c11-4231-a5e6-2169c66b494e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:24:01.193144 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:24:01.230016 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 518/1016] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:24:01 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 15:24:01.253627 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c837c5-d83b-45ec-a4da-caabb7d73081 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:01.257017 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:24:01.362115 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c837c5-d83b-45ec-a4da-caabb7d73081 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:01.399243 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4339e3c1-4e98-4923-9d2b-7afa17077bd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:01.400070 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4339e3c1-4e98-4923-9d2b-7afa17077bd2 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:01.400356 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4339e3c1-4e98-4923-9d2b-7afa17077bd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:01.400631 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4339e3c1-4e98-4923-9d2b-7afa17077bd2 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:01.401285 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4339e3c1-4e98-4923-9d2b-7afa17077bd2 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:01.402648 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4339e3c1-4e98-4923-9d2b-7afa17077bd2 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:01.415383 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-4339e3c1-4e98-4923-9d2b-7afa17077bd2 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 14 15:24:01.415383 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:24:01.415383 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 15:24:01.415383 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 15:24:01.415383 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 15:24:01.415383 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 15:24:01.415383 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 15:24:01.415383 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 15:24:01.415383 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 14 15:24:01.415383 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:24:01.415383 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 15:24:01.415383 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:24:01.415383 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:24:01.415383 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:24:01.415383 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:24:01.415383 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:24:01.415383 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:24:01.416570 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:24:01.416570 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:24:01.416570 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:24:01.416570 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:24:01.416570 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:24:01.416570 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:24:01.416570 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 15:24:01.416570 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 15:24:01.416570 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 15:24:01.416570 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 15:24:01.416570 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:24:01.416570 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:24:01.416570 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:24:01.416570 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:24:01.416570 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:24:01.416570 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:24:01.417410 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:24:01.417410 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:24:01.417410 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:24:01.417410 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:24:01.417410 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:24:01.417410 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:24:01.417410 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:24:01.417410 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:24:01.417410 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:24:01.417410 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:24:01.417410 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 15:24:01.417410 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 15:24:01.417410 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 14 15:24:01.417410 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:24:01.418670 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 499/1017] 10.222.0.99 () {56 vars in 1111 bytes} [Thu Jul 14 15:24:01 2022] GET /identity/v3/services/compute => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:24:01.423292 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d893a11-9c11-4231-a5e6-2169c66b494e None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:01.543268 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d893a11-9c11-4231-a5e6-2169c66b494e None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:01.580537 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194aa9eb-8be8-4fdf-b922-ca260efd031c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:01.581964 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194aa9eb-8be8-4fdf-b922-ca260efd031c None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:01.582513 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194aa9eb-8be8-4fdf-b922-ca260efd031c None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:01.582976 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194aa9eb-8be8-4fdf-b922-ca260efd031c None admin] PATH_INFO: `/v3/services` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:01.583983 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194aa9eb-8be8-4fdf-b922-ca260efd031c None admin] RBAC: Adding query filter params (name=compute) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:24:01.584554 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194aa9eb-8be8-4fdf-b922-ca260efd031c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:01.586123 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194aa9eb-8be8-4fdf-b922-ca260efd031c None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:01.595799 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 519/1018] 10.222.0.99 () {56 vars in 1112 bytes} [Thu Jul 14 15:24:01 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:24:01.598835 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4339e3c1-4e98-4923-9d2b-7afa17077bd2 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:01.655281 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dd7842-2878-4d94-9e0a-b7f3d322278f None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:01.655525 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dd7842-2878-4d94-9e0a-b7f3d322278f None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:01.655804 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dd7842-2878-4d94-9e0a-b7f3d322278f None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:01.657429 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 520/1019] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:24:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:24:01.662273 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1324c9a-a1d5-49bc-bd94-ea2654e1c2da None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:01.662593 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1324c9a-a1d5-49bc-bd94-ea2654e1c2da None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:01.662911 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1324c9a-a1d5-49bc-bd94-ea2654e1c2da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:01.671500 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:24:01.682627 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-d1324c9a-a1d5-49bc-bd94-ea2654e1c2da None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:24:01.683884 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-d1324c9a-a1d5-49bc-bd94-ea2654e1c2da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:24:01.713719 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4339e3c1-4e98-4923-9d2b-7afa17077bd2 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:01.715693 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:24:01.752583 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1697de6-c727-4b24-bcc5-49c750bb4e4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:01.753259 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1697de6-c727-4b24-bcc5-49c750bb4e4b None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:01.753552 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1697de6-c727-4b24-bcc5-49c750bb4e4b None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:01.753835 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1697de6-c727-4b24-bcc5-49c750bb4e4b None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:01.754552 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1697de6-c727-4b24-bcc5-49c750bb4e4b None admin] RBAC: Adding query filter params (type=compute) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:24:01.754868 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1697de6-c727-4b24-bcc5-49c750bb4e4b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:01.756281 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1697de6-c727-4b24-bcc5-49c750bb4e4b None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:01.760697 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 521/1020] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:24:01 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 15:24:01.766925 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 500/1021] 10.222.0.99 () {56 vars in 1112 bytes} [Thu Jul 14 15:24:01 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:24:01.768423 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1324c9a-a1d5-49bc-bd94-ea2654e1c2da None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:01.785747 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1697de6-c727-4b24-bcc5-49c750bb4e4b None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:01.885013 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1324c9a-a1d5-49bc-bd94-ea2654e1c2da None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:01.896459 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1697de6-c727-4b24-bcc5-49c750bb4e4b None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:01.924565 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac0298c-4d99-45d8-b92b-2a98a41fbc55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:01.925460 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac0298c-4d99-45d8-b92b-2a98a41fbc55 None admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:01.925938 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac0298c-4d99-45d8-b92b-2a98a41fbc55 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:01.926402 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac0298c-4d99-45d8-b92b-2a98a41fbc55 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:01.927553 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac0298c-4d99-45d8-b92b-2a98a41fbc55 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:01.929811 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac0298c-4d99-45d8-b92b-2a98a41fbc55 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:01.932534 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:24:01.941606 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5657dd02-f126-4474-aadc-a58f1b31f740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:01.942681 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5657dd02-f126-4474-aadc-a58f1b31f740 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:01.942681 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5657dd02-f126-4474-aadc-a58f1b31f740 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:01.942940 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5657dd02-f126-4474-aadc-a58f1b31f740 None admin] PATH_INFO: `/v3/services/image` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:01.943649 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5657dd02-f126-4474-aadc-a58f1b31f740 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:01.945335 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5657dd02-f126-4474-aadc-a58f1b31f740 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:01.956192 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 522/1022] 10.222.0.99 () {60 vars in 1144 bytes} [Thu Jul 14 15:24:01 2022] POST /identity/v3/endpoints => generated 334 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:24:01.958794 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-5657dd02-f126-4474-aadc-a58f1b31f740 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 14 15:24:01.958794 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:24:01.958794 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 15:24:01.958794 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 15:24:01.958794 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 15:24:01.958794 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 15:24:01.958794 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 15:24:01.958794 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 15:24:01.958794 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 14 15:24:01.958794 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:24:01.958794 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 15:24:01.958794 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:24:01.958794 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:24:01.958794 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:24:01.958794 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:24:01.958794 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:24:01.958794 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:24:01.959830 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:24:01.959830 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:24:01.959830 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:24:01.959830 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:24:01.959830 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:24:01.959830 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:24:01.959830 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 15:24:01.959830 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 15:24:01.959830 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 15:24:01.959830 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 15:24:01.959830 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:24:01.959830 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:24:01.959830 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:24:01.959830 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:24:01.959830 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:24:01.959830 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:24:01.961617 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:24:01.961617 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:24:01.961617 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:24:01.961617 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:24:01.961617 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:24:01.961617 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:24:01.961617 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:24:01.961617 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:24:01.961617 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:24:01.961617 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:24:01.961617 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 15:24:01.961617 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 15:24:01.961617 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 14 15:24:01.961617 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:24:01.961617 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 501/1023] 10.222.0.99 () {56 vars in 1105 bytes} [Thu Jul 14 15:24:01 2022] GET /identity/v3/services/image => generated 86 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 15:24:01.966121 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac0298c-4d99-45d8-b92b-2a98a41fbc55 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:02.000977 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:24:02.100543 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac0298c-4d99-45d8-b92b-2a98a41fbc55 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:02.142033 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb3f4ff-b0a9-4e1e-8ea4-c3a64584ed9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:02.142245 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb3f4ff-b0a9-4e1e-8ea4-c3a64584ed9d None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:02.142503 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb3f4ff-b0a9-4e1e-8ea4-c3a64584ed9d None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:02.142788 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb3f4ff-b0a9-4e1e-8ea4-c3a64584ed9d None admin] PATH_INFO: `/v3/services` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:02.144331 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb3f4ff-b0a9-4e1e-8ea4-c3a64584ed9d None admin] RBAC: Adding query filter params (name=image) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:24:02.144331 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb3f4ff-b0a9-4e1e-8ea4-c3a64584ed9d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:02.145367 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb3f4ff-b0a9-4e1e-8ea4-c3a64584ed9d None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:02.150823 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 523/1024] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:24:01 2022] GET /identity/v3/services?name=image => generated 122 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:24:02.156788 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5657dd02-f126-4474-aadc-a58f1b31f740 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:02.272419 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5657dd02-f126-4474-aadc-a58f1b31f740 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:02.308380 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c27b8c3-a6c3-4bb3-b49f-66e9d529fdf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:02.309053 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c27b8c3-a6c3-4bb3-b49f-66e9d529fdf6 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:02.309346 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c27b8c3-a6c3-4bb3-b49f-66e9d529fdf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:02.309629 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c27b8c3-a6c3-4bb3-b49f-66e9d529fdf6 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:02.310343 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c27b8c3-a6c3-4bb3-b49f-66e9d529fdf6 None admin] RBAC: Adding query filter params (type=image) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:24:02.310655 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c27b8c3-a6c3-4bb3-b49f-66e9d529fdf6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:02.312063 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c27b8c3-a6c3-4bb3-b49f-66e9d529fdf6 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:02.322406 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 502/1025] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:24:02 2022] GET /identity/v3/services?type=image => generated 350 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:24:02.323980 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb3f4ff-b0a9-4e1e-8ea4-c3a64584ed9d None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:02.413251 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:24:02.433547 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb3f4ff-b0a9-4e1e-8ea4-c3a64584ed9d None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:02.467791 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c54403-90fb-4035-b4a8-06cff1e5206f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:02.468457 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c54403-90fb-4035-b4a8-06cff1e5206f None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:02.468768 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c54403-90fb-4035-b4a8-06cff1e5206f None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:02.469055 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c54403-90fb-4035-b4a8-06cff1e5206f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:02.469976 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c54403-90fb-4035-b4a8-06cff1e5206f None admin] RBAC: Adding query filter params (interface=public, service_id=4a482cff43104ff2845c3eb6c7ca2ef1, region_id=RegionOne) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:24:02.470286 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c54403-90fb-4035-b4a8-06cff1e5206f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:02.471685 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c54403-90fb-4035-b4a8-06cff1e5206f None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:02.481372 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 524/1026] 10.222.0.99 () {56 vars in 1251 bytes} [Thu Jul 14 15:24:02 2022] GET /identity/v3/endpoints?service_id=4a482cff43104ff2845c3eb6c7ca2ef1&interface=public®ion_id=RegionOne => generated 194 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:24:02.482654 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c27b8c3-a6c3-4bb3-b49f-66e9d529fdf6 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:02.546277 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:24:02.599731 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c27b8c3-a6c3-4bb3-b49f-66e9d529fdf6 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:02.635458 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad103f69-1897-401b-9ef9-2e0952611251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:02.636169 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad103f69-1897-401b-9ef9-2e0952611251 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:02.636462 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad103f69-1897-401b-9ef9-2e0952611251 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:02.636746 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad103f69-1897-401b-9ef9-2e0952611251 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:02.637687 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad103f69-1897-401b-9ef9-2e0952611251 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:02.639055 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad103f69-1897-401b-9ef9-2e0952611251 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:02.651682 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 503/1027] 10.222.0.99 () {56 vars in 1087 bytes} [Thu Jul 14 15:24:02 2022] GET /identity/v3/services => generated 1957 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:24:03.133728 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbe801b-3b61-4f76-a800-a0048739e213 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:03.134285 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbe801b-3b61-4f76-a800-a0048739e213 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:03.135964 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbe801b-3b61-4f76-a800-a0048739e213 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:03.137862 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 525/1028] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:24:03 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:24:03.144993 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa0c96e-7bef-4f66-9ef5-f4628a56350c None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:03.145302 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa0c96e-7bef-4f66-9ef5-f4628a56350c None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:03.145747 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa0c96e-7bef-4f66-9ef5-f4628a56350c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:03.165255 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-2aa0c96e-7bef-4f66-9ef5-f4628a56350c None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:24:03.166455 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-2aa0c96e-7bef-4f66-9ef5-f4628a56350c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:24:03.238630 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 504/1029] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:24:03 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 15:24:03.263111 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbe801b-3b61-4f76-a800-a0048739e213 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:03.369412 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbe801b-3b61-4f76-a800-a0048739e213 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:03.404576 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ede16b-5b1a-4588-afa3-b750adea2d21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:03.405375 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ede16b-5b1a-4588-afa3-b750adea2d21 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:03.405664 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ede16b-5b1a-4588-afa3-b750adea2d21 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:03.405945 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ede16b-5b1a-4588-afa3-b750adea2d21 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:03.407202 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ede16b-5b1a-4588-afa3-b750adea2d21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:03.408728 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ede16b-5b1a-4588-afa3-b750adea2d21 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:03.415189 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 526/1030] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:24:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:24:03.416258 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa0c96e-7bef-4f66-9ef5-f4628a56350c None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:03.472487 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:24:03.522386 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa0c96e-7bef-4f66-9ef5-f4628a56350c None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:03.557361 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e8e42d-7c47-46e3-af13-712236ee281b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:03.558041 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e8e42d-7c47-46e3-af13-712236ee281b None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:03.558334 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e8e42d-7c47-46e3-af13-712236ee281b None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:03.558618 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e8e42d-7c47-46e3-af13-712236ee281b None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:03.559352 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e8e42d-7c47-46e3-af13-712236ee281b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:03.560823 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e8e42d-7c47-46e3-af13-712236ee281b None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:03.591642 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 505/1031] 10.222.0.99 () {60 vars in 1132 bytes} [Thu Jul 14 15:24:03 2022] POST /identity/v3/users => generated 285 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 15:24:04.147280 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35b9cd6-8ec5-4db3-8373-a86c72ec112c None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:04.147867 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35b9cd6-8ec5-4db3-8373-a86c72ec112c None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:04.148159 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35b9cd6-8ec5-4db3-8373-a86c72ec112c None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:04.149565 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 527/1032] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:24:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:24:04.154521 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b252481-8621-4fc4-84db-855c7afbd1b8 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:04.154920 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b252481-8621-4fc4-84db-855c7afbd1b8 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:04.155080 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b252481-8621-4fc4-84db-855c7afbd1b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:04.173950 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-5b252481-8621-4fc4-84db-855c7afbd1b8 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:24:04.175149 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-5b252481-8621-4fc4-84db-855c7afbd1b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:24:04.247718 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 506/1033] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:24:04 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 15:24:04.271340 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35b9cd6-8ec5-4db3-8373-a86c72ec112c None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:04.289143 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:24:04.379975 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35b9cd6-8ec5-4db3-8373-a86c72ec112c None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:04.417997 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9408f5bf-83dd-405b-ae35-697093ed3ccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:04.418822 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9408f5bf-83dd-405b-ae35-697093ed3ccf None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:04.419111 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9408f5bf-83dd-405b-ae35-697093ed3ccf None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:04.419393 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9408f5bf-83dd-405b-ae35-697093ed3ccf None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:04.431436 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9408f5bf-83dd-405b-ae35-697093ed3ccf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:24:04.439945 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-9408f5bf-83dd-405b-ae35-697093ed3ccf None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 15:24:04.439945 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:24:04.439945 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:24:04.439945 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:24:04.439945 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:24:04.439945 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:24:04.439945 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:24:04.439945 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:24:04.439945 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:24:04.439945 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:24:04.439945 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:24:04.439945 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:24:04.439945 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:24:04.439945 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:24:04.439945 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:24:04.439945 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:24:04.440923 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:24:04.440923 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:24:04.440923 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:24:04.440923 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:24:04.440923 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:24:04.440923 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:24:04.440923 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:24:04.440923 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:24:04.440923 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:24:04.440923 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:24:04.440923 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:24:04.440923 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:24:04.440923 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:24:04.440923 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:24:04.440923 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:24:04.440923 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:24:04.441777 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:24:04.441777 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:24:04.441777 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:24:04.441777 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:24:04.441777 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:24:04.441777 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:24:04.441777 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:24:04.441777 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:24:04.441777 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:24:04.441777 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:24:04.441777 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 15:24:04.441777 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:24:04.441777 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 528/1034] 10.222.0.99 () {56 vars in 1120 bytes} [Thu Jul 14 15:24:04 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:24:04.449590 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b252481-8621-4fc4-84db-855c7afbd1b8 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:04.493052 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:24:04.564218 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b252481-8621-4fc4-84db-855c7afbd1b8 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:04.601402 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7683b841-d7eb-4438-baec-4b878feac559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:04.602099 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7683b841-d7eb-4438-baec-4b878feac559 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:04.602391 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7683b841-d7eb-4438-baec-4b878feac559 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:04.602669 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7683b841-d7eb-4438-baec-4b878feac559 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:04.603418 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7683b841-d7eb-4438-baec-4b878feac559 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:24:04.603757 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7683b841-d7eb-4438-baec-4b878feac559 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:04.605133 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7683b841-d7eb-4438-baec-4b878feac559 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:04.614923 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 507/1035] 10.222.0.99 () {56 vars in 1115 bytes} [Thu Jul 14 15:24:04 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:24:04.619474 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9408f5bf-83dd-405b-ae35-697093ed3ccf None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:04.698673 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:24:04.726116 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9408f5bf-83dd-405b-ae35-697093ed3ccf None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:04.753085 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be367ce7-2a71-4426-9a94-428ec1e69ec7 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:04.753475 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be367ce7-2a71-4426-9a94-428ec1e69ec7 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:04.753656 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be367ce7-2a71-4426-9a94-428ec1e69ec7 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:04.755300 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 508/1036] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:24:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 15:24:04.760090 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3430de1-333c-4c35-994f-ab7bb9bb9f4c None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:04.760389 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3430de1-333c-4c35-994f-ab7bb9bb9f4c None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:04.760678 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3430de1-333c-4c35-994f-ab7bb9bb9f4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:04.767703 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c22573-221d-47df-9e34-14566c29c66a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:04.768733 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c22573-221d-47df-9e34-14566c29c66a None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:04.769162 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c22573-221d-47df-9e34-14566c29c66a None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:04.769581 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c22573-221d-47df-9e34-14566c29c66a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:04.778579 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-c3430de1-333c-4c35-994f-ab7bb9bb9f4c None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:24:04.779871 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c3430de1-333c-4c35-994f-ab7bb9bb9f4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:24:04.782513 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c22573-221d-47df-9e34-14566c29c66a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:04.784340 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c22573-221d-47df-9e34-14566c29c66a None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:04.805845 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-f1c22573-221d-47df-9e34-14566c29c66a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:24:04.805845 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:24:04.805845 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:24:04.805845 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:24:04.805845 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:24:04.805845 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:24:04.805845 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:24:04.805845 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:24:04.805845 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:24:04.805845 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:24:04.805845 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:24:04.805845 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:24:04.805845 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:24:04.805845 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:24:04.805845 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:24:04.807014 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:24:04.807014 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:24:04.807014 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:24:04.807014 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:24:04.807014 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:24:04.807014 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:24:04.807014 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:24:04.807014 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:24:04.807014 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:24:04.807014 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:24:04.807014 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:24:04.807014 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:24:04.807014 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:24:04.807014 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:24:04.807014 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:24:04.807014 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:24:04.807014 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:24:04.808261 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:24:04.808261 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:24:04.808261 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:24:04.808261 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:24:04.811843 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 529/1037] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:24:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:24:04.812728 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c22573-221d-47df-9e34-14566c29c66a None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:04.858209 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 509/1038] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:24:04 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 15:24:04.881995 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3430de1-333c-4c35-994f-ab7bb9bb9f4c None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:04.920146 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:24:04.924549 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c22573-221d-47df-9e34-14566c29c66a None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:04.952983 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:24:04.963327 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6d3b2a-ef40-42d2-90e1-93883b891feb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:04.964043 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6d3b2a-ef40-42d2-90e1-93883b891feb None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:04.964334 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6d3b2a-ef40-42d2-90e1-93883b891feb None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:04.964617 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6d3b2a-ef40-42d2-90e1-93883b891feb None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:04.965346 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6d3b2a-ef40-42d2-90e1-93883b891feb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:24:04.965659 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6d3b2a-ef40-42d2-90e1-93883b891feb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:04.967049 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6d3b2a-ef40-42d2-90e1-93883b891feb None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:04.980195 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 530/1039] 10.222.0.99 () {56 vars in 1109 bytes} [Thu Jul 14 15:24:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:24:04.987306 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6d3b2a-ef40-42d2-90e1-93883b891feb None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:05.004730 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3430de1-333c-4c35-994f-ab7bb9bb9f4c None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:05.043787 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2aa9c6-2c94-4f2f-9cb5-7e6cbd502f62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:05.044584 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2aa9c6-2c94-4f2f-9cb5-7e6cbd502f62 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:05.044875 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2aa9c6-2c94-4f2f-9cb5-7e6cbd502f62 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:05.045164 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2aa9c6-2c94-4f2f-9cb5-7e6cbd502f62 None admin] PATH_INFO: `/v3/services/image` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:05.045846 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2aa9c6-2c94-4f2f-9cb5-7e6cbd502f62 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:05.047273 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2aa9c6-2c94-4f2f-9cb5-7e6cbd502f62 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:05.059273 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-da2aa9c6-2c94-4f2f-9cb5-7e6cbd502f62 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 14 15:24:05.059273 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:24:05.059273 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 15:24:05.059273 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 15:24:05.059273 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 15:24:05.059273 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 15:24:05.059273 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 15:24:05.059273 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 15:24:05.059273 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 14 15:24:05.059273 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:24:05.059273 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 15:24:05.059273 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:24:05.059273 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:24:05.059273 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:24:05.059273 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:24:05.059273 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:24:05.059273 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:24:05.060609 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:24:05.060609 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:24:05.060609 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:24:05.060609 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:24:05.060609 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:24:05.060609 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:24:05.060609 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 15:24:05.060609 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 15:24:05.060609 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 15:24:05.060609 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 15:24:05.060609 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:24:05.060609 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:24:05.060609 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:24:05.060609 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:24:05.060609 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:24:05.060609 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:24:05.061487 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:24:05.061487 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:24:05.061487 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:24:05.061487 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:24:05.061487 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:24:05.061487 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:24:05.061487 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:24:05.061487 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:24:05.061487 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:24:05.061487 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:24:05.061487 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 15:24:05.061487 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 15:24:05.061487 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 14 15:24:05.061487 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:24:05.061487 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 510/1040] 10.222.0.99 () {56 vars in 1105 bytes} [Thu Jul 14 15:24:04 2022] GET /identity/v3/services/image => generated 86 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 15:24:05.067800 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2aa9c6-2c94-4f2f-9cb5-7e6cbd502f62 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:05.106156 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6d3b2a-ef40-42d2-90e1-93883b891feb None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:05.143972 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498bd0d1-a0e0-4873-b852-a1df241046dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:05.144785 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498bd0d1-a0e0-4873-b852-a1df241046dd None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:05.145079 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498bd0d1-a0e0-4873-b852-a1df241046dd None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:05.145365 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498bd0d1-a0e0-4873-b852-a1df241046dd None admin] PATH_INFO: `/v3/users/nova` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:05.155550 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498bd0d1-a0e0-4873-b852-a1df241046dd None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:05.157067 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498bd0d1-a0e0-4873-b852-a1df241046dd None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:05.171628 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-498bd0d1-a0e0-4873-b852-a1df241046dd None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 14 15:24:05.171628 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:24:05.171628 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:24:05.171628 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:24:05.171628 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:24:05.171628 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:24:05.171628 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:24:05.171628 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:24:05.171628 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:24:05.171628 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:24:05.171628 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:24:05.171628 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:24:05.171628 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 15:24:05.171628 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 15:24:05.171628 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 15:24:05.171628 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 15:24:05.173178 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:24:05.173178 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:24:05.173178 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 15:24:05.173178 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:24:05.173178 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 15:24:05.173178 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:24:05.173178 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:24:05.173178 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:24:05.173178 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:24:05.173178 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:24:05.173178 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:24:05.173178 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:24:05.173178 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:24:05.173178 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:24:05.173178 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:24:05.173178 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:24:05.173178 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:24:05.173178 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:24:05.174139 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:24:05.174139 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:24:05.174139 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 15:24:05.174139 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 15:24:05.174139 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 15:24:05.174139 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 15:24:05.174139 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 15:24:05.174139 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 15:24:05.174139 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 14 15:24:05.174139 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:24:05.174139 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 531/1041] 10.222.0.99 () {56 vars in 1093 bytes} [Thu Jul 14 15:24:04 2022] GET /identity/v3/users/nova => generated 82 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 15:24:05.175806 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:24:05.179226 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498bd0d1-a0e0-4873-b852-a1df241046dd None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:05.183929 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2aa9c6-2c94-4f2f-9cb5-7e6cbd502f62 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:05.223061 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d61f67-1785-49eb-b1d0-43be7b3db436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:05.223778 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d61f67-1785-49eb-b1d0-43be7b3db436 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:05.224072 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d61f67-1785-49eb-b1d0-43be7b3db436 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:05.224352 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d61f67-1785-49eb-b1d0-43be7b3db436 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:05.225066 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d61f67-1785-49eb-b1d0-43be7b3db436 None admin] RBAC: Adding query filter params (name=image) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:24:05.226361 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d61f67-1785-49eb-b1d0-43be7b3db436 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:05.227713 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d61f67-1785-49eb-b1d0-43be7b3db436 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:05.231958 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:24:05.234214 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 511/1042] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:24:05 2022] GET /identity/v3/services?name=image => generated 122 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:24:05.240276 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d61f67-1785-49eb-b1d0-43be7b3db436 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:05.294297 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498bd0d1-a0e0-4873-b852-a1df241046dd None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:05.334407 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e9bfd7-15b5-4a8c-9f1b-540f90f4d277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:05.335090 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e9bfd7-15b5-4a8c-9f1b-540f90f4d277 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:05.335383 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e9bfd7-15b5-4a8c-9f1b-540f90f4d277 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:05.335702 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e9bfd7-15b5-4a8c-9f1b-540f90f4d277 None admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:05.336555 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e9bfd7-15b5-4a8c-9f1b-540f90f4d277 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:24:05.336872 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e9bfd7-15b5-4a8c-9f1b-540f90f4d277 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:05.338312 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e9bfd7-15b5-4a8c-9f1b-540f90f4d277 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:05.354888 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d61f67-1785-49eb-b1d0-43be7b3db436 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:05.357429 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 532/1043] 10.222.0.99 () {56 vars in 1133 bytes} [Thu Jul 14 15:24:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:24:05.364765 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e9bfd7-15b5-4a8c-9f1b-540f90f4d277 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:05.398476 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b86695-0fcf-4e33-a802-38c522ca903a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:05.399074 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b86695-0fcf-4e33-a802-38c522ca903a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:05.399391 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b86695-0fcf-4e33-a802-38c522ca903a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:05.399787 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b86695-0fcf-4e33-a802-38c522ca903a None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:05.400624 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b86695-0fcf-4e33-a802-38c522ca903a None admin] RBAC: Adding query filter params (type=image) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:24:05.400943 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b86695-0fcf-4e33-a802-38c522ca903a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:05.402318 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b86695-0fcf-4e33-a802-38c522ca903a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:05.411929 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 512/1044] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:24:05 2022] GET /identity/v3/services?type=image => generated 350 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:24:05.413605 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b86695-0fcf-4e33-a802-38c522ca903a None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:05.441622 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:24:05.469956 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e9bfd7-15b5-4a8c-9f1b-540f90f4d277 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:05.472404 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:24:05.506251 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24f121a-1a88-47fb-9eba-7be2b5a049ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:05.506917 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24f121a-1a88-47fb-9eba-7be2b5a049ef None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:05.507203 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24f121a-1a88-47fb-9eba-7be2b5a049ef None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:05.507484 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24f121a-1a88-47fb-9eba-7be2b5a049ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:05.527380 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b86695-0fcf-4e33-a802-38c522ca903a None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:05.565441 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3618384-d9b8-4aa9-a353-1e38f6eb4652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:05.566117 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3618384-d9b8-4aa9-a353-1e38f6eb4652 None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:05.566409 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3618384-d9b8-4aa9-a353-1e38f6eb4652 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:05.566722 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3618384-d9b8-4aa9-a353-1e38f6eb4652 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:05.567568 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3618384-d9b8-4aa9-a353-1e38f6eb4652 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:05.569295 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3618384-d9b8-4aa9-a353-1e38f6eb4652 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:05.585119 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24f121a-1a88-47fb-9eba-7be2b5a049ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:05.587174 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24f121a-1a88-47fb-9eba-7be2b5a049ef None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:05.593249 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 513/1045] 10.222.0.99 () {60 vars in 1144 bytes} [Thu Jul 14 15:24:05 2022] POST /identity/v3/endpoints => generated 327 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 15:24:05.719372 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 533/1046] 10.222.0.99 () {58 vars in 1282 bytes} [Thu Jul 14 15:24:05 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 14 15:24:05.721535 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3618384-d9b8-4aa9-a353-1e38f6eb4652 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:05.731944 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f7edd5-4c01-4f89-8e6c-aea8b4fa6854 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:05.732280 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f7edd5-4c01-4f89-8e6c-aea8b4fa6854 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:05.732519 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f7edd5-4c01-4f89-8e6c-aea8b4fa6854 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:05.733921 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 534/1047] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:24:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:24:05.739941 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac055db7-a206-41f9-8815-66aa4bcf30ee None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:05.740247 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac055db7-a206-41f9-8815-66aa4bcf30ee None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:05.740512 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac055db7-a206-41f9-8815-66aa4bcf30ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:05.747026 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:24:05.761519 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-ac055db7-a206-41f9-8815-66aa4bcf30ee None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:24:05.763702 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-ac055db7-a206-41f9-8815-66aa4bcf30ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:24:05.824293 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:24:05.853777 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 535/1048] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:24:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 15:24:05.856449 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3618384-d9b8-4aa9-a353-1e38f6eb4652 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:05.877113 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac055db7-a206-41f9-8815-66aa4bcf30ee None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:05.895297 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e13b72b-8129-462c-9a83-fc1ed0fb1d6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:05.896119 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e13b72b-8129-462c-9a83-fc1ed0fb1d6d None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:05.896406 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e13b72b-8129-462c-9a83-fc1ed0fb1d6d None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:05.896689 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e13b72b-8129-462c-9a83-fc1ed0fb1d6d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:05.908493 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e13b72b-8129-462c-9a83-fc1ed0fb1d6d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:05.909985 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e13b72b-8129-462c-9a83-fc1ed0fb1d6d None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:05.928829 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-3e13b72b-8129-462c-9a83-fc1ed0fb1d6d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:24:05.928829 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:24:05.928829 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:24:05.928829 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:24:05.928829 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:24:05.928829 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:24:05.928829 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:24:05.928829 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:24:05.928829 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:24:05.928829 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:24:05.928829 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:24:05.928829 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:24:05.928829 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:24:05.928829 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:24:05.928829 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:24:05.929956 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:24:05.929956 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:24:05.929956 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:24:05.929956 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:24:05.929956 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:24:05.929956 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:24:05.929956 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:24:05.929956 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:24:05.929956 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:24:05.929956 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:24:05.929956 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:24:05.929956 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:24:05.929956 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:24:05.929956 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:24:05.929956 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:24:05.929956 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:24:05.929956 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:24:05.930850 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:24:05.930850 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:24:05.930850 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:24:05.930850 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:24:05.930850 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 514/1049] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:24:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 15:24:05.936418 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e13b72b-8129-462c-9a83-fc1ed0fb1d6d None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:05.997464 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac055db7-a206-41f9-8815-66aa4bcf30ee None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:06.029058 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:24:06.037061 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ec8b32-d5e9-461b-aae1-860f445c1a99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:06.037975 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ec8b32-d5e9-461b-aae1-860f445c1a99 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:06.038271 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ec8b32-d5e9-461b-aae1-860f445c1a99 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:06.038553 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ec8b32-d5e9-461b-aae1-860f445c1a99 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:06.050355 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ec8b32-d5e9-461b-aae1-860f445c1a99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:24:06.058158 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-e4ec8b32-d5e9-461b-aae1-860f445c1a99 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:24:06.058158 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:24:06.058158 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:24:06.058158 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:24:06.058158 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:24:06.058158 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:24:06.058158 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:24:06.058158 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:24:06.058158 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:24:06.058158 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:24:06.058158 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:24:06.058158 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:24:06.058158 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:24:06.058158 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:24:06.058158 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:24:06.058158 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:24:06.059085 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:24:06.059085 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:24:06.059085 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:24:06.059085 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:24:06.059085 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:24:06.059085 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:24:06.059085 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:24:06.059085 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:24:06.059085 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:24:06.059085 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:24:06.059085 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:24:06.059085 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:24:06.059085 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:24:06.059085 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:24:06.059085 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:24:06.059085 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:24:06.059961 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:24:06.059961 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:24:06.059961 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:24:06.059961 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:24:06.059961 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:24:06.059961 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:24:06.059961 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:24:06.059961 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:24:06.059961 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:24:06.059961 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:24:06.059961 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:24:06.059961 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:24:06.059961 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 536/1050] 10.222.0.99 () {56 vars in 1096 bytes} [Thu Jul 14 15:24:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 15:24:06.066309 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ec8b32-d5e9-461b-aae1-860f445c1a99 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:06.075726 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e13b72b-8129-462c-9a83-fc1ed0fb1d6d None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:06.107571 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:24:06.121454 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb8e3d2-4544-4525-b448-10cae4275cb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:06.122394 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb8e3d2-4544-4525-b448-10cae4275cb9 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:06.122684 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb8e3d2-4544-4525-b448-10cae4275cb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:06.122965 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb8e3d2-4544-4525-b448-10cae4275cb9 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:06.123886 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb8e3d2-4544-4525-b448-10cae4275cb9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:24:06.124479 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb8e3d2-4544-4525-b448-10cae4275cb9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:06.126885 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb8e3d2-4544-4525-b448-10cae4275cb9 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:06.145794 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 515/1051] 10.222.0.99 () {56 vars in 1109 bytes} [Thu Jul 14 15:24:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:24:06.150147 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb8e3d2-4544-4525-b448-10cae4275cb9 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:06.187871 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ec8b32-d5e9-461b-aae1-860f445c1a99 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:06.226438 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f97ca15-70dd-4577-829d-cca0ac0b6171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:06.227112 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f97ca15-70dd-4577-829d-cca0ac0b6171 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:06.227397 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f97ca15-70dd-4577-829d-cca0ac0b6171 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:06.227699 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f97ca15-70dd-4577-829d-cca0ac0b6171 None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:06.228448 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f97ca15-70dd-4577-829d-cca0ac0b6171 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:24:06.228752 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f97ca15-70dd-4577-829d-cca0ac0b6171 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:06.230123 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f97ca15-70dd-4577-829d-cca0ac0b6171 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:06.244589 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 537/1052] 10.222.0.99 () {56 vars in 1099 bytes} [Thu Jul 14 15:24:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:24:06.246113 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f97ca15-70dd-4577-829d-cca0ac0b6171 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:06.263379 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb8e3d2-4544-4525-b448-10cae4275cb9 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:06.300225 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e82fb3-a8e6-4f7e-b710-d2c021c20d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:06.301076 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e82fb3-a8e6-4f7e-b710-d2c021c20d1a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:06.301362 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e82fb3-a8e6-4f7e-b710-d2c021c20d1a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:06.301640 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e82fb3-a8e6-4f7e-b710-d2c021c20d1a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:06.308531 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e82fb3-a8e6-4f7e-b710-d2c021c20d1a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:06.308686 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:24:06.309980 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e82fb3-a8e6-4f7e-b710-d2c021c20d1a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:06.324781 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-70e82fb3-a8e6-4f7e-b710-d2c021c20d1a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:24:06.324781 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:24:06.324781 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:24:06.324781 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:24:06.324781 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:24:06.324781 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:24:06.324781 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:24:06.324781 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:24:06.324781 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:24:06.324781 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:24:06.324781 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:24:06.324781 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:24:06.324781 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 15:24:06.324781 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 15:24:06.324781 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 15:24:06.325761 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 15:24:06.325761 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:24:06.325761 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:24:06.325761 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:24:06.325761 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:24:06.325761 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:24:06.325761 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:24:06.325761 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:24:06.325761 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:24:06.325761 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:24:06.325761 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:24:06.325761 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:24:06.325761 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:24:06.325761 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:24:06.325761 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:24:06.325761 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:24:06.325761 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:24:06.326949 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 15:24:06.326949 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 15:24:06.326949 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 15:24:06.326949 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 15:24:06.326949 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 15:24:06.326949 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 15:24:06.326949 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:24:06.326949 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:24:06.326949 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 516/1053] 10.222.0.99 () {56 vars in 1111 bytes} [Thu Jul 14 15:24:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:24:06.331683 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e82fb3-a8e6-4f7e-b710-d2c021c20d1a None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:06.370563 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f97ca15-70dd-4577-829d-cca0ac0b6171 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:06.380781 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:24:06.420276 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f98df20-7b36-4c01-ad99-1765d7917ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:06.421724 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f98df20-7b36-4c01-ad99-1765d7917ef5 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:06.422497 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f98df20-7b36-4c01-ad99-1765d7917ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:06.422962 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f98df20-7b36-4c01-ad99-1765d7917ef5 None admin] PATH_INFO: `/v3/users/14802ce8ea8b4076a66242b6e7b682b0` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:06.439291 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e82fb3-a8e6-4f7e-b710-d2c021c20d1a None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:06.440981 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f98df20-7b36-4c01-ad99-1765d7917ef5 None admin] RBAC: Authorizing `identity:get_user(user_id=14802ce8ea8b4076a66242b6e7b682b0)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:06.442431 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f98df20-7b36-4c01-ad99-1765d7917ef5 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:06.444339 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 538/1054] 10.222.0.99 () {56 vars in 1177 bytes} [Thu Jul 14 15:24:06 2022] GET /identity/v3/users/14802ce8ea8b4076a66242b6e7b682b0 => generated 285 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:24:06.450905 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f98df20-7b36-4c01-ad99-1765d7917ef5 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:06.478328 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8127c6be-cb1d-4cf4-91f9-c399820daf5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:06.479094 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8127c6be-cb1d-4cf4-91f9-c399820daf5f None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:06.479285 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8127c6be-cb1d-4cf4-91f9-c399820daf5f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:06.479570 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8127c6be-cb1d-4cf4-91f9-c399820daf5f None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:06.480468 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8127c6be-cb1d-4cf4-91f9-c399820daf5f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:24:06.480784 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8127c6be-cb1d-4cf4-91f9-c399820daf5f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:06.482247 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8127c6be-cb1d-4cf4-91f9-c399820daf5f None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:06.494872 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 517/1055] 10.222.0.99 () {56 vars in 1148 bytes} [Thu Jul 14 15:24:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:24:06.501277 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8127c6be-cb1d-4cf4-91f9-c399820daf5f None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:06.567721 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f98df20-7b36-4c01-ad99-1765d7917ef5 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:06.582538 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:24:06.610316 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8127c6be-cb1d-4cf4-91f9-c399820daf5f None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:06.610510 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f42cd8-2e53-4bac-a545-f5ef445963b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:06.611180 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f42cd8-2e53-4bac-a545-f5ef445963b7 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:06.611473 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f42cd8-2e53-4bac-a545-f5ef445963b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:06.611790 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f42cd8-2e53-4bac-a545-f5ef445963b7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:06.618834 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:24:06.650369 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70ca1be-d7e5-4c82-a2a0-66de23f7b06d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:06.651038 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70ca1be-d7e5-4c82-a2a0-66de23f7b06d None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:06.651326 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70ca1be-d7e5-4c82-a2a0-66de23f7b06d None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:06.651628 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70ca1be-d7e5-4c82-a2a0-66de23f7b06d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:06.652473 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70ca1be-d7e5-4c82-a2a0-66de23f7b06d None admin] RBAC: Adding query filter params (role.id=ebca1dcd25d64cebaae5f4c18285be4a, scope.project.id=9048d1a6f00443ec92f11471babe5a56, user.id=3a9b1c2622934c5185f3b5f1ea623bc0) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:24:06.652807 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70ca1be-d7e5-4c82-a2a0-66de23f7b06d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:06.654230 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70ca1be-d7e5-4c82-a2a0-66de23f7b06d None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:06.660803 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 518/1056] 10.222.0.99 () {56 vars in 1374 bytes} [Thu Jul 14 15:24:06 2022] GET /identity/v3/role_assignments?user.id=3a9b1c2622934c5185f3b5f1ea623bc0&scope.project.id=9048d1a6f00443ec92f11471babe5a56&role.id=ebca1dcd25d64cebaae5f4c18285be4a => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:24:06.698546 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f42cd8-2e53-4bac-a545-f5ef445963b7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:06.700418 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f42cd8-2e53-4bac-a545-f5ef445963b7 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:06.822089 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 539/1057] 10.222.0.99 () {58 vars in 1282 bytes} [Thu Jul 14 15:24:06 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 14 15:24:06.827571 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70ca1be-d7e5-4c82-a2a0-66de23f7b06d None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:06.936010 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70ca1be-d7e5-4c82-a2a0-66de23f7b06d None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:06.970755 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5d654a-aafc-47e9-b17f-a436d029a283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:06.971666 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5d654a-aafc-47e9-b17f-a436d029a283 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:06.971958 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5d654a-aafc-47e9-b17f-a436d029a283 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:06.972237 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5d654a-aafc-47e9-b17f-a436d029a283 None admin] PATH_INFO: `/v3/projects/1126c60f09584b14a95df33777f31900` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:06.973441 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5d654a-aafc-47e9-b17f-a436d029a283 None admin] RBAC: Authorizing `identity:get_project(project_id=1126c60f09584b14a95df33777f31900)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:06.974898 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5d654a-aafc-47e9-b17f-a436d029a283 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:06.981636 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 519/1058] 10.222.0.99 () {56 vars in 1186 bytes} [Thu Jul 14 15:24:06 2022] GET /identity/v3/projects/1126c60f09584b14a95df33777f31900 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:24:06.983208 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f42cd8-2e53-4bac-a545-f5ef445963b7 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:07.003240 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:24:07.114094 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f42cd8-2e53-4bac-a545-f5ef445963b7 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:07.153350 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5c3519-8229-4728-b536-8c500bb3fbff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:07.154014 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5c3519-8229-4728-b536-8c500bb3fbff None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:07.154299 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5c3519-8229-4728-b536-8c500bb3fbff None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:07.154577 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5c3519-8229-4728-b536-8c500bb3fbff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:07.155437 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5c3519-8229-4728-b536-8c500bb3fbff None admin] RBAC: Adding query filter params (role.id=d0f6ba1314f14d9f9c816bdb5d0c3aac, scope.project.id=1126c60f09584b14a95df33777f31900, user.id=14802ce8ea8b4076a66242b6e7b682b0) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:24:07.155780 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5c3519-8229-4728-b536-8c500bb3fbff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:07.157606 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5c3519-8229-4728-b536-8c500bb3fbff None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:07.163687 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 540/1059] 10.222.0.99 () {56 vars in 1374 bytes} [Thu Jul 14 15:24:06 2022] GET /identity/v3/role_assignments?user.id=14802ce8ea8b4076a66242b6e7b682b0&scope.project.id=1126c60f09584b14a95df33777f31900&role.id=d0f6ba1314f14d9f9c816bdb5d0c3aac => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:24:07.726246 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95105bb4-08f4-4bea-8560-f0000aef4c7d None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:07.726767 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95105bb4-08f4-4bea-8560-f0000aef4c7d None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:07.726883 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95105bb4-08f4-4bea-8560-f0000aef4c7d None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:07.728660 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 520/1060] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:24:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:24:07.735014 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f27c2d9-796e-4885-8dea-34547b9de0a3 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:07.735815 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f27c2d9-796e-4885-8dea-34547b9de0a3 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:07.735815 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f27c2d9-796e-4885-8dea-34547b9de0a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:07.753153 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-4f27c2d9-796e-4885-8dea-34547b9de0a3 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:24:07.754400 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-4f27c2d9-796e-4885-8dea-34547b9de0a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:24:07.831468 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 541/1061] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:24:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 15:24:07.854889 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95105bb4-08f4-4bea-8560-f0000aef4c7d None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:07.899321 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:24:07.965891 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95105bb4-08f4-4bea-8560-f0000aef4c7d None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:08.010011 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41674ff-a78a-4bfe-b828-bdcf2eea2c67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:08.010673 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41674ff-a78a-4bfe-b828-bdcf2eea2c67 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:08.010963 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41674ff-a78a-4bfe-b828-bdcf2eea2c67 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:08.011245 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41674ff-a78a-4bfe-b828-bdcf2eea2c67 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:08.082752 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41674ff-a78a-4bfe-b828-bdcf2eea2c67 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:08.084260 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41674ff-a78a-4bfe-b828-bdcf2eea2c67 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:08.122336 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:24:08.188948 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 521/1062] 10.222.0.99 () {58 vars in 1282 bytes} [Thu Jul 14 15:24:07 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 15:24:08.194626 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f27c2d9-796e-4885-8dea-34547b9de0a3 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:08.197787 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:24:08.305239 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f27c2d9-796e-4885-8dea-34547b9de0a3 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:08.340909 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ed7e3f-ab6a-447d-b646-2fd500dd0c28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:08.341710 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ed7e3f-ab6a-447d-b646-2fd500dd0c28 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:08.341996 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ed7e3f-ab6a-447d-b646-2fd500dd0c28 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:08.342273 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ed7e3f-ab6a-447d-b646-2fd500dd0c28 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:08.352723 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ed7e3f-ab6a-447d-b646-2fd500dd0c28 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:08.354188 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ed7e3f-ab6a-447d-b646-2fd500dd0c28 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:08.372415 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-73ed7e3f-ab6a-447d-b646-2fd500dd0c28 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:24:08.372415 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:24:08.372415 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:24:08.372415 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:24:08.372415 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:24:08.372415 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:24:08.372415 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:24:08.372415 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:24:08.372415 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:24:08.372415 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:24:08.372415 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:24:08.372415 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:24:08.372415 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:24:08.372415 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:24:08.372415 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:24:08.373401 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:24:08.373401 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:24:08.373401 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:24:08.373401 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:24:08.373401 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:24:08.373401 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:24:08.373401 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:24:08.373401 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:24:08.373401 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:24:08.373401 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:24:08.373401 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:24:08.373401 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:24:08.373401 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:24:08.373401 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:24:08.373401 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:24:08.373401 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:24:08.373401 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:24:08.374304 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:24:08.374304 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:24:08.374304 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:24:08.374304 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:24:08.376139 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 542/1063] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:24:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:24:08.380446 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41674ff-a78a-4bfe-b828-bdcf2eea2c67 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:08.480905 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41674ff-a78a-4bfe-b828-bdcf2eea2c67 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:08.517760 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6007a4-72db-454e-b68d-341ac3cd5085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:08.518429 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6007a4-72db-454e-b68d-341ac3cd5085 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:08.518714 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6007a4-72db-454e-b68d-341ac3cd5085 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:08.518995 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6007a4-72db-454e-b68d-341ac3cd5085 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:08.519829 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6007a4-72db-454e-b68d-341ac3cd5085 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:24:08.520101 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6007a4-72db-454e-b68d-341ac3cd5085 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:08.521427 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6007a4-72db-454e-b68d-341ac3cd5085 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:08.528887 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:24:08.535850 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 522/1064] 10.222.0.99 () {56 vars in 1109 bytes} [Thu Jul 14 15:24:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:24:08.766125 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6bbb88-b20c-4bee-8fc6-6d7a4ac198f2 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:08.766695 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6bbb88-b20c-4bee-8fc6-6d7a4ac198f2 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:08.766695 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6bbb88-b20c-4bee-8fc6-6d7a4ac198f2 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:08.768055 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 543/1065] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:24:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:24:08.773637 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36eb0af-bfb1-4464-b03f-33ff6c25930d None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:08.774099 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36eb0af-bfb1-4464-b03f-33ff6c25930d None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:08.774303 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36eb0af-bfb1-4464-b03f-33ff6c25930d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:08.793076 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-c36eb0af-bfb1-4464-b03f-33ff6c25930d None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:24:08.794280 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c36eb0af-bfb1-4464-b03f-33ff6c25930d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:24:08.871027 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 523/1066] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:24:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 15:24:08.895923 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6bbb88-b20c-4bee-8fc6-6d7a4ac198f2 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:08.953721 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:24:09.006570 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6bbb88-b20c-4bee-8fc6-6d7a4ac198f2 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:09.040772 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197226fa-0522-45cb-928d-2b7183f97278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:09.041509 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197226fa-0522-45cb-928d-2b7183f97278 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:09.041799 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197226fa-0522-45cb-928d-2b7183f97278 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:09.042080 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197226fa-0522-45cb-928d-2b7183f97278 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:09.052725 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197226fa-0522-45cb-928d-2b7183f97278 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:24:09.061055 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-197226fa-0522-45cb-928d-2b7183f97278 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 15:24:09.061055 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:24:09.061055 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:24:09.061055 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:24:09.061055 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:24:09.061055 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:24:09.061055 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:24:09.061055 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:24:09.061055 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:24:09.061055 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:24:09.061055 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:24:09.061055 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:24:09.061055 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:24:09.061055 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:24:09.061055 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:24:09.061055 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:24:09.062229 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:24:09.062229 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:24:09.062229 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:24:09.062229 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:24:09.062229 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:24:09.062229 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:24:09.062229 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:24:09.062229 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:24:09.062229 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:24:09.062229 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:24:09.062229 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:24:09.062229 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:24:09.062229 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:24:09.062229 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:24:09.062229 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:24:09.062229 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:24:09.064148 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:24:09.064148 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:24:09.064148 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:24:09.064148 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:24:09.064148 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:24:09.064148 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:24:09.064148 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:24:09.064148 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:24:09.064148 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:24:09.064148 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:24:09.064148 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 15:24:09.064148 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:24:09.067742 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 544/1067] 10.222.0.99 () {56 vars in 1120 bytes} [Thu Jul 14 15:24:08 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:24:09.069192 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36eb0af-bfb1-4464-b03f-33ff6c25930d None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:09.171577 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36eb0af-bfb1-4464-b03f-33ff6c25930d None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:09.202309 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:24:09.211469 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8642979-831e-4f53-a91b-a9625714f123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:09.212243 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8642979-831e-4f53-a91b-a9625714f123 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:09.212532 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8642979-831e-4f53-a91b-a9625714f123 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:09.212813 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8642979-831e-4f53-a91b-a9625714f123 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:09.213579 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8642979-831e-4f53-a91b-a9625714f123 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:24:09.213892 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8642979-831e-4f53-a91b-a9625714f123 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:09.215293 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8642979-831e-4f53-a91b-a9625714f123 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:09.224060 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 524/1068] 10.222.0.99 () {56 vars in 1115 bytes} [Thu Jul 14 15:24:09 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:24:09.229826 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197226fa-0522-45cb-928d-2b7183f97278 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:09.240650 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ea5312-250e-4ee6-889f-35096bcd31fb None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:09.240981 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ea5312-250e-4ee6-889f-35096bcd31fb None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:09.241202 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ea5312-250e-4ee6-889f-35096bcd31fb None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:09.242569 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 525/1069] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:24:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 15:24:09.247508 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e47afe2-fc97-4302-b7e2-95490d6c0683 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:09.247824 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e47afe2-fc97-4302-b7e2-95490d6c0683 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:09.248156 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e47afe2-fc97-4302-b7e2-95490d6c0683 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:09.265372 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-6e47afe2-fc97-4302-b7e2-95490d6c0683 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:24:09.266826 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-6e47afe2-fc97-4302-b7e2-95490d6c0683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:24:09.334500 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 526/1070] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:24:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 15:24:09.336201 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197226fa-0522-45cb-928d-2b7183f97278 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:09.353133 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:24:09.358691 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e47afe2-fc97-4302-b7e2-95490d6c0683 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:09.373405 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7db0288-7a8c-4d7d-8523-62cb0e1c2c47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:09.374199 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7db0288-7a8c-4d7d-8523-62cb0e1c2c47 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:09.374483 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7db0288-7a8c-4d7d-8523-62cb0e1c2c47 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:09.374757 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7db0288-7a8c-4d7d-8523-62cb0e1c2c47 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:09.388469 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7db0288-7a8c-4d7d-8523-62cb0e1c2c47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:09.389954 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7db0288-7a8c-4d7d-8523-62cb0e1c2c47 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:09.408230 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-c7db0288-7a8c-4d7d-8523-62cb0e1c2c47 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:24:09.408230 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:24:09.408230 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:24:09.408230 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:24:09.408230 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:24:09.408230 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:24:09.408230 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:24:09.408230 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:24:09.408230 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:24:09.408230 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:24:09.408230 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:24:09.408230 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:24:09.408230 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:24:09.408230 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:24:09.408230 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:24:09.409348 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:24:09.409348 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:24:09.409348 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:24:09.409348 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:24:09.409348 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:24:09.409348 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:24:09.409348 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:24:09.409348 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:24:09.409348 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:24:09.409348 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:24:09.409348 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:24:09.409348 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:24:09.409348 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:24:09.409348 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:24:09.409348 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:24:09.409348 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:24:09.409348 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:24:09.410259 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:24:09.410259 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:24:09.410259 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:24:09.410259 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:24:09.410259 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 545/1071] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:24:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 15:24:09.415207 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7db0288-7a8c-4d7d-8523-62cb0e1c2c47 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:09.461897 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e47afe2-fc97-4302-b7e2-95490d6c0683 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:09.482911 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:24:09.497711 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15a9a6e-f9b1-4083-a5d9-90c6332980f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:09.498506 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15a9a6e-f9b1-4083-a5d9-90c6332980f5 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:09.498788 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15a9a6e-f9b1-4083-a5d9-90c6332980f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:09.499062 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15a9a6e-f9b1-4083-a5d9-90c6332980f5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:09.509354 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15a9a6e-f9b1-4083-a5d9-90c6332980f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:24:09.516006 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7db0288-7a8c-4d7d-8523-62cb0e1c2c47 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:09.517735 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-e15a9a6e-f9b1-4083-a5d9-90c6332980f5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:24:09.517735 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:24:09.517735 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:24:09.517735 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:24:09.517735 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:24:09.517735 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:24:09.517735 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:24:09.517735 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:24:09.517735 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:24:09.517735 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:24:09.517735 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:24:09.517735 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:24:09.517735 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:24:09.517735 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:24:09.517735 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:24:09.517735 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:24:09.518688 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:24:09.518688 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:24:09.518688 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:24:09.518688 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:24:09.518688 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:24:09.518688 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:24:09.518688 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:24:09.518688 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:24:09.518688 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:24:09.518688 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:24:09.518688 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:24:09.518688 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:24:09.518688 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:24:09.518688 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:24:09.518688 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:24:09.518688 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:24:09.519922 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:24:09.519922 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:24:09.519922 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:24:09.519922 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:24:09.519922 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:24:09.519922 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:24:09.519922 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:24:09.519922 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:24:09.519922 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:24:09.519922 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:24:09.519922 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:24:09.519922 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:24:09.519922 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 527/1072] 10.222.0.99 () {56 vars in 1096 bytes} [Thu Jul 14 15:24:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 15:24:09.524926 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15a9a6e-f9b1-4083-a5d9-90c6332980f5 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:09.551804 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542eb04a-5d37-4f78-afa1-b18ef2462d9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:09.552481 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542eb04a-5d37-4f78-afa1-b18ef2462d9f None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:09.552765 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542eb04a-5d37-4f78-afa1-b18ef2462d9f None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:09.553045 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542eb04a-5d37-4f78-afa1-b18ef2462d9f None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:09.553778 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542eb04a-5d37-4f78-afa1-b18ef2462d9f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:24:09.554084 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542eb04a-5d37-4f78-afa1-b18ef2462d9f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:09.555612 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542eb04a-5d37-4f78-afa1-b18ef2462d9f None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:09.577010 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 546/1073] 10.222.0.99 () {56 vars in 1109 bytes} [Thu Jul 14 15:24:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:24:09.578065 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542eb04a-5d37-4f78-afa1-b18ef2462d9f None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:09.606103 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:24:09.638095 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15a9a6e-f9b1-4083-a5d9-90c6332980f5 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:09.671925 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126ce99e-3b55-479f-9e06-08eada8068f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:09.673181 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126ce99e-3b55-479f-9e06-08eada8068f9 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:09.673474 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126ce99e-3b55-479f-9e06-08eada8068f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:09.673757 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126ce99e-3b55-479f-9e06-08eada8068f9 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:09.674508 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126ce99e-3b55-479f-9e06-08eada8068f9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:24:09.674819 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126ce99e-3b55-479f-9e06-08eada8068f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:09.676223 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126ce99e-3b55-479f-9e06-08eada8068f9 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:09.678258 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542eb04a-5d37-4f78-afa1-b18ef2462d9f None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:09.685325 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 528/1074] 10.222.0.99 () {56 vars in 1099 bytes} [Thu Jul 14 15:24:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:24:09.691330 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126ce99e-3b55-479f-9e06-08eada8068f9 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:09.714054 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46f54b7-c36f-4223-a6f1-f32a50425404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:09.714846 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46f54b7-c36f-4223-a6f1-f32a50425404 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:09.715127 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46f54b7-c36f-4223-a6f1-f32a50425404 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:09.715405 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46f54b7-c36f-4223-a6f1-f32a50425404 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:09.722631 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46f54b7-c36f-4223-a6f1-f32a50425404 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:09.724114 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46f54b7-c36f-4223-a6f1-f32a50425404 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:09.735705 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:24:09.740520 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-e46f54b7-c36f-4223-a6f1-f32a50425404 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 14 15:24:09.740520 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:24:09.740520 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:24:09.740520 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:24:09.740520 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:24:09.740520 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:24:09.740520 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:24:09.740520 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:24:09.740520 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:24:09.740520 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:24:09.740520 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:24:09.740520 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:24:09.740520 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 15:24:09.740520 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 15:24:09.740520 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 15:24:09.740520 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 15:24:09.741531 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:24:09.741531 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:24:09.741531 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 15:24:09.741531 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:24:09.741531 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 15:24:09.741531 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:24:09.741531 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:24:09.741531 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:24:09.741531 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:24:09.741531 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:24:09.741531 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:24:09.741531 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:24:09.741531 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:24:09.741531 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:24:09.741531 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:24:09.741531 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:24:09.741531 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:24:09.741531 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:24:09.742707 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:24:09.742707 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:24:09.742707 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 15:24:09.742707 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 15:24:09.742707 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 15:24:09.742707 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 15:24:09.742707 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 15:24:09.742707 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 15:24:09.742707 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 14 15:24:09.742707 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:24:09.742707 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 547/1075] 10.222.0.99 () {56 vars in 1093 bytes} [Thu Jul 14 15:24:09 2022] GET /identity/v3/users/nova => generated 82 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:24:09.747692 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46f54b7-c36f-4223-a6f1-f32a50425404 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:09.798802 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126ce99e-3b55-479f-9e06-08eada8068f9 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:09.833036 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d5454c-4f43-418e-8d0e-f63e3da04c08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:09.833902 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d5454c-4f43-418e-8d0e-f63e3da04c08 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:09.834186 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d5454c-4f43-418e-8d0e-f63e3da04c08 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:09.834462 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d5454c-4f43-418e-8d0e-f63e3da04c08 None admin] PATH_INFO: `/v3/users/14802ce8ea8b4076a66242b6e7b682b0` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:09.836229 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d5454c-4f43-418e-8d0e-f63e3da04c08 None admin] RBAC: Authorizing `identity:get_user(user_id=14802ce8ea8b4076a66242b6e7b682b0)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:09.837693 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d5454c-4f43-418e-8d0e-f63e3da04c08 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:09.839369 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 529/1076] 10.222.0.99 () {56 vars in 1177 bytes} [Thu Jul 14 15:24:09 2022] GET /identity/v3/users/14802ce8ea8b4076a66242b6e7b682b0 => generated 285 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:24:09.846308 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:24:09.846781 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d5454c-4f43-418e-8d0e-f63e3da04c08 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:09.858701 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46f54b7-c36f-4223-a6f1-f32a50425404 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:09.895257 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9b2b84-5bea-48f7-8d47-79358f3ac18c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:09.895962 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9b2b84-5bea-48f7-8d47-79358f3ac18c None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:09.896253 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9b2b84-5bea-48f7-8d47-79358f3ac18c None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:09.896530 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9b2b84-5bea-48f7-8d47-79358f3ac18c None admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:09.897378 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9b2b84-5bea-48f7-8d47-79358f3ac18c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:24:09.897689 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9b2b84-5bea-48f7-8d47-79358f3ac18c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:09.899127 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9b2b84-5bea-48f7-8d47-79358f3ac18c None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:09.917160 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 548/1077] 10.222.0.99 () {56 vars in 1133 bytes} [Thu Jul 14 15:24:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:24:09.923617 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9b2b84-5bea-48f7-8d47-79358f3ac18c None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:09.950056 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d5454c-4f43-418e-8d0e-f63e3da04c08 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:09.960453 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:24:09.985022 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d3a773-ad7c-454b-ba31-4b06d0448b35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:09.985881 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d3a773-ad7c-454b-ba31-4b06d0448b35 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:09.986173 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d3a773-ad7c-454b-ba31-4b06d0448b35 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:09.989460 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d3a773-ad7c-454b-ba31-4b06d0448b35 None admin] PATH_INFO: `/v3/projects/1126c60f09584b14a95df33777f31900` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:09.990333 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d3a773-ad7c-454b-ba31-4b06d0448b35 None admin] RBAC: Authorizing `identity:get_project(project_id=1126c60f09584b14a95df33777f31900)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:09.991662 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d3a773-ad7c-454b-ba31-4b06d0448b35 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:09.993642 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 530/1078] 10.222.0.99 () {56 vars in 1186 bytes} [Thu Jul 14 15:24:09 2022] GET /identity/v3/projects/1126c60f09584b14a95df33777f31900 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:24:09.999647 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d3a773-ad7c-454b-ba31-4b06d0448b35 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:10.025505 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9b2b84-5bea-48f7-8d47-79358f3ac18c None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:10.059258 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf32823-5949-45ba-9dab-3cb3048f55f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:10.060073 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf32823-5949-45ba-9dab-3cb3048f55f5 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:10.060354 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf32823-5949-45ba-9dab-3cb3048f55f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:10.060632 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf32823-5949-45ba-9dab-3cb3048f55f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:10.071282 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf32823-5949-45ba-9dab-3cb3048f55f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:10.072742 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf32823-5949-45ba-9dab-3cb3048f55f5 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:10.079840 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:24:10.091456 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-7cf32823-5949-45ba-9dab-3cb3048f55f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:24:10.091456 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:24:10.091456 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:24:10.091456 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:24:10.091456 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:24:10.091456 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:24:10.091456 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:24:10.091456 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:24:10.091456 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:24:10.091456 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:24:10.091456 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:24:10.091456 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:24:10.091456 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:24:10.091456 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:24:10.091456 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:24:10.092486 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:24:10.092486 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:24:10.092486 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:24:10.092486 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:24:10.092486 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:24:10.092486 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:24:10.092486 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:24:10.092486 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:24:10.092486 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:24:10.092486 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:24:10.092486 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:24:10.092486 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:24:10.092486 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:24:10.092486 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:24:10.092486 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:24:10.092486 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:24:10.092486 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:24:10.093368 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:24:10.093368 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:24:10.093368 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:24:10.093368 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:24:10.093368 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 549/1079] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:24:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:24:10.098997 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf32823-5949-45ba-9dab-3cb3048f55f5 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:10.106732 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d3a773-ad7c-454b-ba31-4b06d0448b35 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:10.143223 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbe31be-909e-48ce-b9ad-4af44cf77bcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:10.144153 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbe31be-909e-48ce-b9ad-4af44cf77bcb None admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:10.144445 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbe31be-909e-48ce-b9ad-4af44cf77bcb None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:10.144730 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbe31be-909e-48ce-b9ad-4af44cf77bcb None admin] PATH_INFO: `/v3/projects/1126c60f09584b14a95df33777f31900/users/14802ce8ea8b4076a66242b6e7b682b0/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:10.146734 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbe31be-909e-48ce-b9ad-4af44cf77bcb None admin] RBAC: Authorizing `identity:create_grant(project_id=1126c60f09584b14a95df33777f31900, user_id=14802ce8ea8b4076a66242b6e7b682b0, role_id=d0f6ba1314f14d9f9c816bdb5d0c3aac)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:10.148204 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbe31be-909e-48ce-b9ad-4af44cf77bcb None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:10.161597 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 531/1080] 10.222.0.99 () {58 vars in 1439 bytes} [Thu Jul 14 15:24:09 2022] PUT /identity/v3/projects/1126c60f09584b14a95df33777f31900/users/14802ce8ea8b4076a66242b6e7b682b0/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 15:24:10.205250 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf32823-5949-45ba-9dab-3cb3048f55f5 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:10.244558 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e82a0c-6276-4f4a-97ca-f623d4e1d8b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:10.245252 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e82a0c-6276-4f4a-97ca-f623d4e1d8b2 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:10.245542 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e82a0c-6276-4f4a-97ca-f623d4e1d8b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:10.245822 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e82a0c-6276-4f4a-97ca-f623d4e1d8b2 None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:10.246564 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e82a0c-6276-4f4a-97ca-f623d4e1d8b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:24:10.246875 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e82a0c-6276-4f4a-97ca-f623d4e1d8b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:10.248325 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e82a0c-6276-4f4a-97ca-f623d4e1d8b2 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:10.261627 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 550/1081] 10.222.0.99 () {56 vars in 1109 bytes} [Thu Jul 14 15:24:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:24:10.267739 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbe31be-909e-48ce-b9ad-4af44cf77bcb None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:10.314632 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:24:10.376116 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbe31be-909e-48ce-b9ad-4af44cf77bcb None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:10.410931 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e08579a-5302-43c2-9c7a-b7dd2d171f19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:10.411679 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e08579a-5302-43c2-9c7a-b7dd2d171f19 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:10.411972 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e08579a-5302-43c2-9c7a-b7dd2d171f19 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:10.412253 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e08579a-5302-43c2-9c7a-b7dd2d171f19 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:10.419052 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e08579a-5302-43c2-9c7a-b7dd2d171f19 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:10.420528 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e08579a-5302-43c2-9c7a-b7dd2d171f19 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:10.435706 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-0e08579a-5302-43c2-9c7a-b7dd2d171f19 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:24:10.435706 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:24:10.435706 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:24:10.435706 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:24:10.435706 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:24:10.435706 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:24:10.435706 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:24:10.435706 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:24:10.435706 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:24:10.435706 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:24:10.435706 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:24:10.435706 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:24:10.435706 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 15:24:10.435706 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 15:24:10.435706 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 15:24:10.436716 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 15:24:10.436716 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:24:10.436716 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:24:10.436716 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:24:10.436716 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:24:10.436716 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:24:10.436716 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:24:10.436716 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:24:10.436716 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:24:10.436716 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:24:10.436716 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:24:10.436716 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:24:10.436716 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:24:10.436716 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:24:10.436716 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:24:10.436716 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:24:10.436716 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:24:10.437656 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 15:24:10.437656 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 15:24:10.437656 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 15:24:10.437656 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 15:24:10.437656 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 15:24:10.437656 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 15:24:10.437656 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:24:10.437656 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:24:10.437656 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 532/1082] 10.222.0.99 () {56 vars in 1111 bytes} [Thu Jul 14 15:24:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:24:10.443336 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e82a0c-6276-4f4a-97ca-f623d4e1d8b2 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:10.502455 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:24:10.545438 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e82a0c-6276-4f4a-97ca-f623d4e1d8b2 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:10.579407 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5a717f-884a-4947-9223-b34f7f076e74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:10.580118 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5a717f-884a-4947-9223-b34f7f076e74 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:10.580411 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5a717f-884a-4947-9223-b34f7f076e74 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:10.580694 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5a717f-884a-4947-9223-b34f7f076e74 None admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:10.581496 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5a717f-884a-4947-9223-b34f7f076e74 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:24:10.581813 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5a717f-884a-4947-9223-b34f7f076e74 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:10.583260 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5a717f-884a-4947-9223-b34f7f076e74 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:10.601070 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 551/1083] 10.222.0.99 () {56 vars in 1148 bytes} [Thu Jul 14 15:24:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:24:10.607159 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e08579a-5302-43c2-9c7a-b7dd2d171f19 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:10.709466 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:24:10.713846 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e08579a-5302-43c2-9c7a-b7dd2d171f19 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:10.746994 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd63eb5-a77a-4e8d-a7ab-3f3ccbd9ebe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:10.747893 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd63eb5-a77a-4e8d-a7ab-3f3ccbd9ebe6 None admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:10.748179 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd63eb5-a77a-4e8d-a7ab-3f3ccbd9ebe6 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:10.748456 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd63eb5-a77a-4e8d-a7ab-3f3ccbd9ebe6 None admin] PATH_INFO: `/v3/projects/9048d1a6f00443ec92f11471babe5a56/users/3a9b1c2622934c5185f3b5f1ea623bc0/roles/ebca1dcd25d64cebaae5f4c18285be4a` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:10.750699 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd63eb5-a77a-4e8d-a7ab-3f3ccbd9ebe6 None admin] RBAC: Authorizing `identity:create_grant(project_id=9048d1a6f00443ec92f11471babe5a56, user_id=3a9b1c2622934c5185f3b5f1ea623bc0, role_id=ebca1dcd25d64cebaae5f4c18285be4a)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:10.752151 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd63eb5-a77a-4e8d-a7ab-3f3ccbd9ebe6 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:10.765150 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 533/1084] 10.222.0.99 () {58 vars in 1439 bytes} [Thu Jul 14 15:24:10 2022] PUT /identity/v3/projects/9048d1a6f00443ec92f11471babe5a56/users/3a9b1c2622934c5185f3b5f1ea623bc0/roles/ebca1dcd25d64cebaae5f4c18285be4a => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:24:12.223556 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e2dc28-7717-4b4d-a46b-fbb41ed9faff None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:12.224133 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e2dc28-7717-4b4d-a46b-fbb41ed9faff None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:12.224133 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e2dc28-7717-4b4d-a46b-fbb41ed9faff None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:12.225515 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 552/1085] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:24:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:24:12.230826 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da53390e-c75a-4747-9f9c-80ef17e942d2 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:12.231253 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da53390e-c75a-4747-9f9c-80ef17e942d2 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:12.231400 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da53390e-c75a-4747-9f9c-80ef17e942d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:12.249308 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-da53390e-c75a-4747-9f9c-80ef17e942d2 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:24:12.251571 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-da53390e-c75a-4747-9f9c-80ef17e942d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:24:12.319798 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 534/1086] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:24:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 15:24:12.344265 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e2dc28-7717-4b4d-a46b-fbb41ed9faff None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:12.463830 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e2dc28-7717-4b4d-a46b-fbb41ed9faff None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:12.481348 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:24:12.498905 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c979e18-78d9-447c-a38d-d2742d94fcfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:12.499805 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c979e18-78d9-447c-a38d-d2742d94fcfd None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:12.500538 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c979e18-78d9-447c-a38d-d2742d94fcfd None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:12.500808 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c979e18-78d9-447c-a38d-d2742d94fcfd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:12.510886 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c979e18-78d9-447c-a38d-d2742d94fcfd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:24:12.519352 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-3c979e18-78d9-447c-a38d-d2742d94fcfd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:24:12.519352 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:24:12.519352 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:24:12.519352 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:24:12.519352 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:24:12.519352 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:24:12.519352 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:24:12.519352 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:24:12.519352 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:24:12.519352 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:24:12.519352 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:24:12.519352 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:24:12.519352 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:24:12.519352 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:24:12.519352 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:24:12.519352 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:24:12.520624 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:24:12.520624 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:24:12.520624 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:24:12.520624 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:24:12.520624 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:24:12.520624 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:24:12.520624 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:24:12.520624 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:24:12.520624 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:24:12.520624 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:24:12.520624 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:24:12.520624 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:24:12.520624 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:24:12.520624 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:24:12.520624 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:24:12.520624 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:24:12.521486 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:24:12.521486 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:24:12.521486 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:24:12.521486 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:24:12.521486 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:24:12.521486 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:24:12.521486 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:24:12.521486 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:24:12.521486 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:24:12.521486 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:24:12.521486 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:24:12.521486 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:24:12.521486 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 553/1087] 10.222.0.99 () {56 vars in 1096 bytes} [Thu Jul 14 15:24:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:24:12.527057 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da53390e-c75a-4747-9f9c-80ef17e942d2 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:12.623168 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:24:12.642999 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da53390e-c75a-4747-9f9c-80ef17e942d2 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:12.683570 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18448f7-f55c-40ef-83cd-935a5a986a41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:12.684631 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18448f7-f55c-40ef-83cd-935a5a986a41 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:12.684920 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18448f7-f55c-40ef-83cd-935a5a986a41 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:12.685201 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18448f7-f55c-40ef-83cd-935a5a986a41 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:12.685958 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18448f7-f55c-40ef-83cd-935a5a986a41 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:24:12.686267 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18448f7-f55c-40ef-83cd-935a5a986a41 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:12.688453 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18448f7-f55c-40ef-83cd-935a5a986a41 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:12.698931 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 535/1088] 10.222.0.99 () {56 vars in 1099 bytes} [Thu Jul 14 15:24:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:24:12.706456 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c979e18-78d9-447c-a38d-d2742d94fcfd None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:12.842114 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c979e18-78d9-447c-a38d-d2742d94fcfd None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:12.879266 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1eb6a1-fd56-40bb-a938-45cee545900f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:12.879679 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71d5e26-c068-45e2-a7bb-1b18ba1f943d None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:12.879977 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71d5e26-c068-45e2-a7bb-1b18ba1f943d None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:12.880262 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71d5e26-c068-45e2-a7bb-1b18ba1f943d None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:12.880776 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1eb6a1-fd56-40bb-a938-45cee545900f None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:12.881247 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1eb6a1-fd56-40bb-a938-45cee545900f None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:12.881540 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1eb6a1-fd56-40bb-a938-45cee545900f None admin] PATH_INFO: `/v3/users/14802ce8ea8b4076a66242b6e7b682b0` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:12.881884 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 536/1089] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:24:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 15:24:12.882903 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1eb6a1-fd56-40bb-a938-45cee545900f None admin] RBAC: Authorizing `identity:get_user(user_id=14802ce8ea8b4076a66242b6e7b682b0)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:12.884419 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1eb6a1-fd56-40bb-a938-45cee545900f None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:12.887054 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 554/1090] 10.222.0.99 () {56 vars in 1177 bytes} [Thu Jul 14 15:24:12 2022] GET /identity/v3/users/14802ce8ea8b4076a66242b6e7b682b0 => generated 285 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:24:12.888179 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb35bf7-3f5f-47c3-9c48-b138265ab5c7 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:12.888477 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb35bf7-3f5f-47c3-9c48-b138265ab5c7 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:12.888768 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb35bf7-3f5f-47c3-9c48-b138265ab5c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:12.892666 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1eb6a1-fd56-40bb-a938-45cee545900f None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:12.907988 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-fcb35bf7-3f5f-47c3-9c48-b138265ab5c7 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:24:12.909303 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-fcb35bf7-3f5f-47c3-9c48-b138265ab5c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:24:12.945951 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:24:12.987317 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 537/1091] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:24:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 15:24:13.014075 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1eb6a1-fd56-40bb-a938-45cee545900f None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:13.014428 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb35bf7-3f5f-47c3-9c48-b138265ab5c7 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:13.051437 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f85c5e0-3ce0-4b44-8b88-81098acc4e30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:13.052140 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f85c5e0-3ce0-4b44-8b88-81098acc4e30 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:13.052803 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f85c5e0-3ce0-4b44-8b88-81098acc4e30 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:13.053092 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f85c5e0-3ce0-4b44-8b88-81098acc4e30 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:13.109400 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:24:13.131002 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb35bf7-3f5f-47c3-9c48-b138265ab5c7 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:13.134280 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f85c5e0-3ce0-4b44-8b88-81098acc4e30 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:13.135789 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f85c5e0-3ce0-4b44-8b88-81098acc4e30 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:13.168834 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b7ffd1-efc6-4802-9c8c-4911de7c12f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:13.169654 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b7ffd1-efc6-4802-9c8c-4911de7c12f5 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:13.169942 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b7ffd1-efc6-4802-9c8c-4911de7c12f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:13.170220 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b7ffd1-efc6-4802-9c8c-4911de7c12f5 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:13.181232 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b7ffd1-efc6-4802-9c8c-4911de7c12f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:24:13.190784 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-08b7ffd1-efc6-4802-9c8c-4911de7c12f5 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 15:24:13.190784 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:24:13.190784 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:24:13.190784 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:24:13.190784 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:24:13.190784 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:24:13.190784 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:24:13.190784 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:24:13.190784 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:24:13.190784 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:24:13.190784 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:24:13.190784 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:24:13.190784 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:24:13.190784 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:24:13.190784 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:24:13.190784 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:24:13.191860 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:24:13.191860 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:24:13.191860 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:24:13.191860 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:24:13.191860 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:24:13.191860 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:24:13.191860 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:24:13.191860 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:24:13.191860 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:24:13.191860 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:24:13.191860 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:24:13.191860 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:24:13.191860 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:24:13.191860 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:24:13.191860 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:24:13.191860 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:24:13.192967 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:24:13.192967 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:24:13.192967 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:24:13.192967 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:24:13.192967 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:24:13.192967 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:24:13.192967 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:24:13.192967 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:24:13.192967 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:24:13.192967 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:24:13.192967 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 15:24:13.192967 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:24:13.192967 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 538/1092] 10.222.0.99 () {56 vars in 1120 bytes} [Thu Jul 14 15:24:13 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 15:24:13.193847 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:24:13.199459 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b7ffd1-efc6-4802-9c8c-4911de7c12f5 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:13.257163 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 555/1093] 10.222.0.99 () {58 vars in 1282 bytes} [Thu Jul 14 15:24:12 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 14 15:24:13.263611 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f85c5e0-3ce0-4b44-8b88-81098acc4e30 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:13.313636 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b7ffd1-efc6-4802-9c8c-4911de7c12f5 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:13.348883 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d68b31c-6e28-46ef-b8fa-43cbdaf52c64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:13.349555 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d68b31c-6e28-46ef-b8fa-43cbdaf52c64 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:13.349856 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d68b31c-6e28-46ef-b8fa-43cbdaf52c64 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:13.350216 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d68b31c-6e28-46ef-b8fa-43cbdaf52c64 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:13.351006 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d68b31c-6e28-46ef-b8fa-43cbdaf52c64 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:24:13.351403 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d68b31c-6e28-46ef-b8fa-43cbdaf52c64 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:13.352885 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d68b31c-6e28-46ef-b8fa-43cbdaf52c64 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:13.361513 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 539/1094] 10.222.0.99 () {56 vars in 1115 bytes} [Thu Jul 14 15:24:13 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:24:13.367457 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d68b31c-6e28-46ef-b8fa-43cbdaf52c64 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:13.370452 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:24:13.381547 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f85c5e0-3ce0-4b44-8b88-81098acc4e30 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:13.416836 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19ee09e-672c-4904-b610-9c8828022db7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:13.417694 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19ee09e-672c-4904-b610-9c8828022db7 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:13.417986 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19ee09e-672c-4904-b610-9c8828022db7 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:13.418261 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19ee09e-672c-4904-b610-9c8828022db7 None admin] PATH_INFO: `/v3/projects/1126c60f09584b14a95df33777f31900` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:13.419446 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19ee09e-672c-4904-b610-9c8828022db7 None admin] RBAC: Authorizing `identity:get_project(project_id=1126c60f09584b14a95df33777f31900)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:13.420927 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19ee09e-672c-4904-b610-9c8828022db7 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:13.424710 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 556/1095] 10.222.0.99 () {56 vars in 1186 bytes} [Thu Jul 14 15:24:13 2022] GET /identity/v3/projects/1126c60f09584b14a95df33777f31900 => generated 311 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:24:13.430915 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19ee09e-672c-4904-b610-9c8828022db7 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:13.447255 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:24:13.472979 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d68b31c-6e28-46ef-b8fa-43cbdaf52c64 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:13.507096 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1f3eaa-9354-42ba-93c0-2c255ecd70ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:13.507917 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1f3eaa-9354-42ba-93c0-2c255ecd70ed None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:13.508200 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1f3eaa-9354-42ba-93c0-2c255ecd70ed None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:13.508473 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1f3eaa-9354-42ba-93c0-2c255ecd70ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:13.519522 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1f3eaa-9354-42ba-93c0-2c255ecd70ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:13.520997 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1f3eaa-9354-42ba-93c0-2c255ecd70ed None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:13.533198 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19ee09e-672c-4904-b610-9c8828022db7 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:13.537906 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-8f1f3eaa-9354-42ba-93c0-2c255ecd70ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:24:13.537906 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:24:13.537906 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:24:13.537906 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:24:13.537906 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:24:13.537906 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:24:13.537906 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:24:13.537906 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:24:13.537906 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:24:13.537906 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:24:13.537906 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:24:13.537906 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:24:13.537906 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:24:13.537906 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:24:13.537906 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:24:13.539168 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:24:13.539168 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:24:13.539168 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:24:13.539168 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:24:13.539168 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:24:13.539168 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:24:13.539168 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:24:13.539168 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:24:13.539168 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:24:13.539168 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:24:13.539168 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:24:13.539168 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:24:13.539168 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:24:13.539168 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:24:13.539168 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:24:13.539168 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:24:13.539168 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:24:13.540219 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:24:13.540219 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:24:13.540219 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:24:13.540219 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:24:13.540219 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 540/1096] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:24:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 15:24:13.544933 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1f3eaa-9354-42ba-93c0-2c255ecd70ed None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:13.569927 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7c9743-628d-479b-881c-0a9d7a3d2565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:13.570579 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7c9743-628d-479b-881c-0a9d7a3d2565 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:13.571020 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7c9743-628d-479b-881c-0a9d7a3d2565 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:13.571317 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7c9743-628d-479b-881c-0a9d7a3d2565 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:13.572240 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7c9743-628d-479b-881c-0a9d7a3d2565 None admin] RBAC: Adding query filter params (role.id=d0f6ba1314f14d9f9c816bdb5d0c3aac, scope.project.id=1126c60f09584b14a95df33777f31900, user.id=14802ce8ea8b4076a66242b6e7b682b0) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:24:13.572501 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7c9743-628d-479b-881c-0a9d7a3d2565 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:13.574058 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7c9743-628d-479b-881c-0a9d7a3d2565 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:13.580244 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 557/1097] 10.222.0.99 () {56 vars in 1374 bytes} [Thu Jul 14 15:24:13 2022] GET /identity/v3/role_assignments?user.id=14802ce8ea8b4076a66242b6e7b682b0&scope.project.id=1126c60f09584b14a95df33777f31900&role.id=d0f6ba1314f14d9f9c816bdb5d0c3aac => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:24:13.612628 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:24:13.664118 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1f3eaa-9354-42ba-93c0-2c255ecd70ed None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:13.704493 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d73c2be-bca7-4949-86fe-aaee82144dbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:13.705005 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d73c2be-bca7-4949-86fe-aaee82144dbe None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:13.705296 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d73c2be-bca7-4949-86fe-aaee82144dbe None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:13.705580 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d73c2be-bca7-4949-86fe-aaee82144dbe None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:13.706319 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d73c2be-bca7-4949-86fe-aaee82144dbe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:24:13.706661 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d73c2be-bca7-4949-86fe-aaee82144dbe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:13.708108 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d73c2be-bca7-4949-86fe-aaee82144dbe None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:13.722006 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 541/1098] 10.222.0.99 () {56 vars in 1109 bytes} [Thu Jul 14 15:24:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:24:13.728105 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7c9743-628d-479b-881c-0a9d7a3d2565 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:13.809436 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:24:13.832340 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7c9743-628d-479b-881c-0a9d7a3d2565 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:13.866038 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cf9eb7-2094-4042-8c8b-88ab4e30ebd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:13.866754 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cf9eb7-2094-4042-8c8b-88ab4e30ebd0 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:13.867041 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cf9eb7-2094-4042-8c8b-88ab4e30ebd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:13.867321 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cf9eb7-2094-4042-8c8b-88ab4e30ebd0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:13.876178 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cf9eb7-2094-4042-8c8b-88ab4e30ebd0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:13.877627 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cf9eb7-2094-4042-8c8b-88ab4e30ebd0 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:13.892900 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-01cf9eb7-2094-4042-8c8b-88ab4e30ebd0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 14 15:24:13.892900 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:24:13.892900 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:24:13.892900 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:24:13.892900 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:24:13.892900 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:24:13.892900 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:24:13.892900 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:24:13.892900 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:24:13.892900 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:24:13.892900 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:24:13.892900 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:24:13.892900 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 15:24:13.892900 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 15:24:13.892900 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 15:24:13.892900 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 15:24:13.894031 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:24:13.894031 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:24:13.894031 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 15:24:13.894031 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:24:13.894031 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 15:24:13.894031 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:24:13.894031 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:24:13.894031 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:24:13.894031 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:24:13.894031 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:24:13.894031 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:24:13.894031 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:24:13.894031 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:24:13.894031 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:24:13.894031 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:24:13.894031 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:24:13.894031 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:24:13.894031 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:24:13.894980 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:24:13.894980 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:24:13.894980 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 15:24:13.894980 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 15:24:13.894980 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 15:24:13.894980 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 15:24:13.894980 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 15:24:13.894980 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 15:24:13.894980 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 14 15:24:13.894980 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:24:13.894980 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 558/1099] 10.222.0.99 () {56 vars in 1093 bytes} [Thu Jul 14 15:24:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 15:24:13.899823 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d73c2be-bca7-4949-86fe-aaee82144dbe None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:14.004441 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d73c2be-bca7-4949-86fe-aaee82144dbe None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:14.019450 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:24:14.040193 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75837fe9-b48b-4a97-a727-8c9a249cf644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:14.040892 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75837fe9-b48b-4a97-a727-8c9a249cf644 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:14.041182 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75837fe9-b48b-4a97-a727-8c9a249cf644 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:14.041462 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75837fe9-b48b-4a97-a727-8c9a249cf644 None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:14.042300 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75837fe9-b48b-4a97-a727-8c9a249cf644 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:24:14.042615 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75837fe9-b48b-4a97-a727-8c9a249cf644 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:14.044111 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75837fe9-b48b-4a97-a727-8c9a249cf644 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:14.061498 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 542/1100] 10.222.0.99 () {56 vars in 1133 bytes} [Thu Jul 14 15:24:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:24:14.068093 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cf9eb7-2094-4042-8c8b-88ab4e30ebd0 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:14.183250 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cf9eb7-2094-4042-8c8b-88ab4e30ebd0 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:14.217484 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e9868e-4dbb-4ab8-9443-ffd188e7aada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:14.218132 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e9868e-4dbb-4ab8-9443-ffd188e7aada None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:14.218422 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e9868e-4dbb-4ab8-9443-ffd188e7aada None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:14.218769 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e9868e-4dbb-4ab8-9443-ffd188e7aada None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:14.226896 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:24:14.287049 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e9868e-4dbb-4ab8-9443-ffd188e7aada None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:14.288577 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e9868e-4dbb-4ab8-9443-ffd188e7aada None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:14.385654 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 559/1101] 10.222.0.99 () {58 vars in 1282 bytes} [Thu Jul 14 15:24:14 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 14 15:24:14.391442 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75837fe9-b48b-4a97-a727-8c9a249cf644 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:14.488879 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75837fe9-b48b-4a97-a727-8c9a249cf644 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:14.522196 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a24ebb-118b-4dd3-bf7a-5410d8758371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:14.522973 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a24ebb-118b-4dd3-bf7a-5410d8758371 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:14.523258 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a24ebb-118b-4dd3-bf7a-5410d8758371 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:14.523535 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a24ebb-118b-4dd3-bf7a-5410d8758371 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:14.533863 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a24ebb-118b-4dd3-bf7a-5410d8758371 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:14.535290 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a24ebb-118b-4dd3-bf7a-5410d8758371 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:14.552080 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-06a24ebb-118b-4dd3-bf7a-5410d8758371 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:24:14.552080 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:24:14.552080 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:24:14.552080 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:24:14.552080 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:24:14.552080 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:24:14.552080 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:24:14.552080 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:24:14.552080 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:24:14.552080 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:24:14.552080 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:24:14.552080 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:24:14.552080 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:24:14.552080 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:24:14.552080 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:24:14.553132 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:24:14.553132 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:24:14.553132 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:24:14.553132 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:24:14.553132 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:24:14.553132 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:24:14.553132 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:24:14.553132 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:24:14.553132 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:24:14.553132 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:24:14.553132 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:24:14.553132 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:24:14.553132 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:24:14.553132 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:24:14.553132 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:24:14.553132 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:24:14.553132 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:24:14.554190 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:24:14.554190 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:24:14.554190 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:24:14.554190 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:24:14.554190 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 543/1102] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:24:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:24:14.559088 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e9868e-4dbb-4ab8-9443-ffd188e7aada None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:14.599034 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:24:14.661470 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e9868e-4dbb-4ab8-9443-ffd188e7aada None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:14.698225 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e50e5d-3940-4583-8c37-a24e85049306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:14.698897 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e50e5d-3940-4583-8c37-a24e85049306 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:14.699193 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e50e5d-3940-4583-8c37-a24e85049306 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:14.699477 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e50e5d-3940-4583-8c37-a24e85049306 None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:14.700243 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e50e5d-3940-4583-8c37-a24e85049306 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:24:14.700557 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e50e5d-3940-4583-8c37-a24e85049306 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:14.701927 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e50e5d-3940-4583-8c37-a24e85049306 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:14.714281 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 560/1103] 10.222.0.99 () {56 vars in 1109 bytes} [Thu Jul 14 15:24:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:24:14.720329 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a24ebb-118b-4dd3-bf7a-5410d8758371 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:14.747129 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:24:14.823386 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a24ebb-118b-4dd3-bf7a-5410d8758371 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:14.858714 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a465cc-0635-4d0c-a1f2-429781429da9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:14.859983 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a465cc-0635-4d0c-a1f2-429781429da9 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:14.860273 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a465cc-0635-4d0c-a1f2-429781429da9 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:14.860553 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a465cc-0635-4d0c-a1f2-429781429da9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:14.867121 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a465cc-0635-4d0c-a1f2-429781429da9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:14.868607 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a465cc-0635-4d0c-a1f2-429781429da9 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:14.882889 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-40a465cc-0635-4d0c-a1f2-429781429da9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:24:14.882889 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:24:14.882889 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:24:14.882889 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:24:14.882889 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:24:14.882889 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:24:14.882889 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:24:14.882889 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:24:14.882889 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:24:14.882889 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:24:14.882889 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:24:14.882889 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:24:14.882889 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 15:24:14.882889 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 15:24:14.882889 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 15:24:14.883780 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 15:24:14.883780 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:24:14.883780 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:24:14.883780 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:24:14.883780 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:24:14.883780 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:24:14.883780 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:24:14.883780 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:24:14.883780 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:24:14.883780 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:24:14.883780 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:24:14.883780 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:24:14.883780 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:24:14.883780 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:24:14.883780 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:24:14.883780 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:24:14.883780 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:24:14.885260 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 15:24:14.885260 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 15:24:14.885260 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 15:24:14.885260 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 15:24:14.885260 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 15:24:14.885260 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 15:24:14.885260 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:24:14.885260 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:24:14.885260 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 544/1104] 10.222.0.99 () {56 vars in 1111 bytes} [Thu Jul 14 15:24:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 15:24:14.890220 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e50e5d-3940-4583-8c37-a24e85049306 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:14.988908 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:24:14.994848 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e50e5d-3940-4583-8c37-a24e85049306 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:15.030322 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84243a7-136f-47e3-b4d1-4a9b6b9255ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:15.031988 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84243a7-136f-47e3-b4d1-4a9b6b9255ea None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:15.032518 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84243a7-136f-47e3-b4d1-4a9b6b9255ea None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:15.033026 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84243a7-136f-47e3-b4d1-4a9b6b9255ea None admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:15.034450 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84243a7-136f-47e3-b4d1-4a9b6b9255ea None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:24:15.035006 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84243a7-136f-47e3-b4d1-4a9b6b9255ea None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:15.036909 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84243a7-136f-47e3-b4d1-4a9b6b9255ea None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:15.049281 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 561/1105] 10.222.0.99 () {56 vars in 1148 bytes} [Thu Jul 14 15:24:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:24:15.055840 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a465cc-0635-4d0c-a1f2-429781429da9 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:15.141754 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:24:15.158282 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a465cc-0635-4d0c-a1f2-429781429da9 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:15.191686 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1485cb56-c57b-4382-9bb6-52120012a541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:15.192343 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1485cb56-c57b-4382-9bb6-52120012a541 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:15.192634 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1485cb56-c57b-4382-9bb6-52120012a541 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:15.192918 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1485cb56-c57b-4382-9bb6-52120012a541 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:15.193757 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1485cb56-c57b-4382-9bb6-52120012a541 None admin] RBAC: Adding query filter params (role.id=ebca1dcd25d64cebaae5f4c18285be4a, scope.project.id=9048d1a6f00443ec92f11471babe5a56, user.id=3a9b1c2622934c5185f3b5f1ea623bc0) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:24:15.194078 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1485cb56-c57b-4382-9bb6-52120012a541 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:15.195512 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1485cb56-c57b-4382-9bb6-52120012a541 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:15.201452 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 545/1106] 10.222.0.99 () {56 vars in 1374 bytes} [Thu Jul 14 15:24:15 2022] GET /identity/v3/role_assignments?user.id=3a9b1c2622934c5185f3b5f1ea623bc0&scope.project.id=9048d1a6f00443ec92f11471babe5a56&role.id=ebca1dcd25d64cebaae5f4c18285be4a => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:24:15.666114 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9098f598-1f0d-43f3-9985-c43086fd101b None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:15.666632 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9098f598-1f0d-43f3-9985-c43086fd101b None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:15.666752 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9098f598-1f0d-43f3-9985-c43086fd101b None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:15.668104 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 562/1107] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:24:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:24:15.674008 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8d6cef-2d50-4474-ab83-3c8a02ac01dd None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:15.674457 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8d6cef-2d50-4474-ab83-3c8a02ac01dd None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:15.674548 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8d6cef-2d50-4474-ab83-3c8a02ac01dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:15.693275 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-6a8d6cef-2d50-4474-ab83-3c8a02ac01dd None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:24:15.694498 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-6a8d6cef-2d50-4474-ab83-3c8a02ac01dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:24:15.764453 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 546/1108] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:24:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 15:24:15.788580 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9098f598-1f0d-43f3-9985-c43086fd101b None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:15.898927 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9098f598-1f0d-43f3-9985-c43086fd101b None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:15.937131 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbda2a2-6b61-4fc2-9358-1e32d67ff652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:15.938643 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbda2a2-6b61-4fc2-9358-1e32d67ff652 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:15.938919 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbda2a2-6b61-4fc2-9358-1e32d67ff652 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:15.939220 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbda2a2-6b61-4fc2-9358-1e32d67ff652 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:15.940599 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbda2a2-6b61-4fc2-9358-1e32d67ff652 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:15.942048 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbda2a2-6b61-4fc2-9358-1e32d67ff652 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:15.944669 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 563/1109] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:24:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:24:15.949766 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8d6cef-2d50-4474-ab83-3c8a02ac01dd None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:16.035718 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:24:16.059147 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8d6cef-2d50-4474-ab83-3c8a02ac01dd None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:16.093291 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058121a3-4ba7-4ba9-89f0-f6969b763c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:16.094102 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058121a3-4ba7-4ba9-89f0-f6969b763c5b None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:16.094388 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058121a3-4ba7-4ba9-89f0-f6969b763c5b None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:16.094664 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058121a3-4ba7-4ba9-89f0-f6969b763c5b None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:16.095376 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058121a3-4ba7-4ba9-89f0-f6969b763c5b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:16.096845 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058121a3-4ba7-4ba9-89f0-f6969b763c5b None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:16.116925 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 547/1110] 10.222.0.99 () {60 vars in 1132 bytes} [Thu Jul 14 15:24:15 2022] POST /identity/v3/users => generated 286 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:24:18.190440 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b782b045-f7f8-483f-82d4-9fc1b3f5a51e None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:18.190991 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b782b045-f7f8-483f-82d4-9fc1b3f5a51e None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:18.190991 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b782b045-f7f8-483f-82d4-9fc1b3f5a51e None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:18.192382 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 564/1111] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:24:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:24:18.197521 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3887e028-148a-49cd-83d0-323a772e1403 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:18.197904 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3887e028-148a-49cd-83d0-323a772e1403 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:18.198151 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3887e028-148a-49cd-83d0-323a772e1403 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:18.215901 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-3887e028-148a-49cd-83d0-323a772e1403 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:24:18.216910 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-3887e028-148a-49cd-83d0-323a772e1403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:24:18.285363 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 548/1112] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:24:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 15:24:18.309037 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b782b045-f7f8-483f-82d4-9fc1b3f5a51e None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:18.338279 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:24:18.413391 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b782b045-f7f8-483f-82d4-9fc1b3f5a51e None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:18.447012 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9762361-054f-4b81-9961-d238219e9a38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:18.447895 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9762361-054f-4b81-9961-d238219e9a38 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:18.448184 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9762361-054f-4b81-9961-d238219e9a38 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:18.448465 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9762361-054f-4b81-9961-d238219e9a38 None admin] PATH_INFO: `/v3/roles/3627b4f45c8c4f41a7def6b15f1b8225` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:18.449946 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9762361-054f-4b81-9961-d238219e9a38 None admin] RBAC: Authorizing `identity:get_role(role_id=3627b4f45c8c4f41a7def6b15f1b8225)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:18.451324 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9762361-054f-4b81-9961-d238219e9a38 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:18.452948 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 565/1113] 10.222.0.99 () {56 vars in 1177 bytes} [Thu Jul 14 15:24:18 2022] GET /identity/v3/roles/3627b4f45c8c4f41a7def6b15f1b8225 => generated 223 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:24:18.459147 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3887e028-148a-49cd-83d0-323a772e1403 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:18.520590 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:24:18.561345 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3887e028-148a-49cd-83d0-323a772e1403 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:18.595178 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2da59a-414f-4a81-b5c4-8cbe67f53f8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:18.596074 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2da59a-414f-4a81-b5c4-8cbe67f53f8e None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:18.596364 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2da59a-414f-4a81-b5c4-8cbe67f53f8e None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:18.596643 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2da59a-414f-4a81-b5c4-8cbe67f53f8e None admin] PATH_INFO: `/v3/users/672dba645ae244648d10e7a37e115b82` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:18.610255 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2da59a-414f-4a81-b5c4-8cbe67f53f8e None admin] RBAC: Authorizing `identity:get_user(user_id=672dba645ae244648d10e7a37e115b82)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:18.611715 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2da59a-414f-4a81-b5c4-8cbe67f53f8e None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:18.613368 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 549/1114] 10.222.0.99 () {56 vars in 1177 bytes} [Thu Jul 14 15:24:18 2022] GET /identity/v3/users/672dba645ae244648d10e7a37e115b82 => generated 286 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:24:18.619268 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9762361-054f-4b81-9961-d238219e9a38 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:18.712708 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:24:18.720009 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9762361-054f-4b81-9961-d238219e9a38 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:18.755847 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6cceee-a46e-42df-b11c-3c01cb0cfc28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:18.756500 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6cceee-a46e-42df-b11c-3c01cb0cfc28 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:18.756785 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6cceee-a46e-42df-b11c-3c01cb0cfc28 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:18.757065 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6cceee-a46e-42df-b11c-3c01cb0cfc28 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:18.834864 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6cceee-a46e-42df-b11c-3c01cb0cfc28 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:18.836262 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6cceee-a46e-42df-b11c-3c01cb0cfc28 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:18.929192 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:24:18.936793 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 566/1115] 10.222.0.99 () {58 vars in 1282 bytes} [Thu Jul 14 15:24:18 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 15:24:18.943311 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2da59a-414f-4a81-b5c4-8cbe67f53f8e None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:19.039502 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2da59a-414f-4a81-b5c4-8cbe67f53f8e None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:19.061489 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:24:19.073485 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67397f5f-742a-40fd-b05e-9c191d7de8ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:19.074330 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67397f5f-742a-40fd-b05e-9c191d7de8ae None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:19.074611 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67397f5f-742a-40fd-b05e-9c191d7de8ae None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:19.074887 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67397f5f-742a-40fd-b05e-9c191d7de8ae None admin] PATH_INFO: `/v3/projects/1126c60f09584b14a95df33777f31900` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:19.076066 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67397f5f-742a-40fd-b05e-9c191d7de8ae None admin] RBAC: Authorizing `identity:get_project(project_id=1126c60f09584b14a95df33777f31900)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:19.077477 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67397f5f-742a-40fd-b05e-9c191d7de8ae None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:19.079323 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 550/1116] 10.222.0.99 () {56 vars in 1186 bytes} [Thu Jul 14 15:24:18 2022] GET /identity/v3/projects/1126c60f09584b14a95df33777f31900 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:24:19.085573 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6cceee-a46e-42df-b11c-3c01cb0cfc28 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:19.189632 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6cceee-a46e-42df-b11c-3c01cb0cfc28 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:19.223880 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900807e2-0a2a-401e-99a7-7f6a519e2c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:19.224535 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900807e2-0a2a-401e-99a7-7f6a519e2c11 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:19.224825 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900807e2-0a2a-401e-99a7-7f6a519e2c11 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:19.225106 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900807e2-0a2a-401e-99a7-7f6a519e2c11 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:19.225944 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900807e2-0a2a-401e-99a7-7f6a519e2c11 None admin] RBAC: Adding query filter params (role.id=3627b4f45c8c4f41a7def6b15f1b8225, scope.project.id=1126c60f09584b14a95df33777f31900, user.id=672dba645ae244648d10e7a37e115b82) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:24:19.226258 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900807e2-0a2a-401e-99a7-7f6a519e2c11 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:19.227717 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900807e2-0a2a-401e-99a7-7f6a519e2c11 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:19.233062 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 567/1117] 10.222.0.99 () {56 vars in 1374 bytes} [Thu Jul 14 15:24:19 2022] GET /identity/v3/role_assignments?user.id=672dba645ae244648d10e7a37e115b82&scope.project.id=1126c60f09584b14a95df33777f31900&role.id=3627b4f45c8c4f41a7def6b15f1b8225 => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:24:21.285288 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9553f4-2136-4bbc-962e-28301c02320c None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:21.285814 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9553f4-2136-4bbc-962e-28301c02320c None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:21.285814 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9553f4-2136-4bbc-962e-28301c02320c None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:21.287136 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 551/1118] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:24:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:24:21.292374 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da2c6dc-ac42-4bab-aea8-c4ee667836d6 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:21.292901 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da2c6dc-ac42-4bab-aea8-c4ee667836d6 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:21.293007 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da2c6dc-ac42-4bab-aea8-c4ee667836d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:21.309300 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-2da2c6dc-ac42-4bab-aea8-c4ee667836d6 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:24:21.310478 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-2da2c6dc-ac42-4bab-aea8-c4ee667836d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:24:21.362177 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:24:21.376132 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 568/1119] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:24:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 15:24:21.398703 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9553f4-2136-4bbc-962e-28301c02320c None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:21.498014 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9553f4-2136-4bbc-962e-28301c02320c None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:21.530628 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b032db49-5ef4-405f-9daf-b8fd9935a734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:21.532025 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b032db49-5ef4-405f-9daf-b8fd9935a734 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:21.532309 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b032db49-5ef4-405f-9daf-b8fd9935a734 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:21.532583 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b032db49-5ef4-405f-9daf-b8fd9935a734 None admin] PATH_INFO: `/v3/roles/3627b4f45c8c4f41a7def6b15f1b8225` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:21.534044 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b032db49-5ef4-405f-9daf-b8fd9935a734 None admin] RBAC: Authorizing `identity:get_role(role_id=3627b4f45c8c4f41a7def6b15f1b8225)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:21.535451 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b032db49-5ef4-405f-9daf-b8fd9935a734 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:21.536794 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 552/1120] 10.222.0.99 () {56 vars in 1177 bytes} [Thu Jul 14 15:24:21 2022] GET /identity/v3/roles/3627b4f45c8c4f41a7def6b15f1b8225 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:24:21.542640 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da2c6dc-ac42-4bab-aea8-c4ee667836d6 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:21.639086 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da2c6dc-ac42-4bab-aea8-c4ee667836d6 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:21.672857 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9ab7dc-010c-45d0-bbad-089de22a71ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:21.673720 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9ab7dc-010c-45d0-bbad-089de22a71ee None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:21.674006 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9ab7dc-010c-45d0-bbad-089de22a71ee None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:21.674283 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9ab7dc-010c-45d0-bbad-089de22a71ee None admin] PATH_INFO: `/v3/users/672dba645ae244648d10e7a37e115b82` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:21.675452 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9ab7dc-010c-45d0-bbad-089de22a71ee None admin] RBAC: Authorizing `identity:get_user(user_id=672dba645ae244648d10e7a37e115b82)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:21.676917 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9ab7dc-010c-45d0-bbad-089de22a71ee None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:21.678516 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 569/1121] 10.222.0.99 () {56 vars in 1177 bytes} [Thu Jul 14 15:24:21 2022] GET /identity/v3/users/672dba645ae244648d10e7a37e115b82 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:24:21.684782 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b032db49-5ef4-405f-9daf-b8fd9935a734 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:21.743155 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:24:21.786196 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b032db49-5ef4-405f-9daf-b8fd9935a734 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:21.819479 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11e7a31-04d4-41d7-9e44-b0985e8e016e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:21.820366 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11e7a31-04d4-41d7-9e44-b0985e8e016e None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:21.820654 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11e7a31-04d4-41d7-9e44-b0985e8e016e None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:21.820935 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11e7a31-04d4-41d7-9e44-b0985e8e016e None admin] PATH_INFO: `/v3/projects/1126c60f09584b14a95df33777f31900` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:21.822112 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11e7a31-04d4-41d7-9e44-b0985e8e016e None admin] RBAC: Authorizing `identity:get_project(project_id=1126c60f09584b14a95df33777f31900)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:21.823537 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11e7a31-04d4-41d7-9e44-b0985e8e016e None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:21.825262 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 553/1122] 10.222.0.99 () {56 vars in 1186 bytes} [Thu Jul 14 15:24:21 2022] GET /identity/v3/projects/1126c60f09584b14a95df33777f31900 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:24:21.832036 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9ab7dc-010c-45d0-bbad-089de22a71ee None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:21.886653 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:24:21.932228 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9ab7dc-010c-45d0-bbad-089de22a71ee None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:21.965860 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdbb1b7-92b6-41af-8613-e9bad377852c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:21.966749 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdbb1b7-92b6-41af-8613-e9bad377852c None admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:21.967041 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdbb1b7-92b6-41af-8613-e9bad377852c None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:21.967322 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdbb1b7-92b6-41af-8613-e9bad377852c None admin] PATH_INFO: `/v3/projects/1126c60f09584b14a95df33777f31900/users/672dba645ae244648d10e7a37e115b82/roles/3627b4f45c8c4f41a7def6b15f1b8225` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:21.971101 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdbb1b7-92b6-41af-8613-e9bad377852c None admin] RBAC: Authorizing `identity:create_grant(project_id=1126c60f09584b14a95df33777f31900, user_id=672dba645ae244648d10e7a37e115b82, role_id=3627b4f45c8c4f41a7def6b15f1b8225)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:21.972569 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdbb1b7-92b6-41af-8613-e9bad377852c None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:21.985976 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 570/1123] 10.222.0.99 () {58 vars in 1439 bytes} [Thu Jul 14 15:24:21 2022] PUT /identity/v3/projects/1126c60f09584b14a95df33777f31900/users/672dba645ae244648d10e7a37e115b82/roles/3627b4f45c8c4f41a7def6b15f1b8225 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 15:24:24.041911 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcf149e-fa40-4819-b8eb-e28cd209c451 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:24.042406 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcf149e-fa40-4819-b8eb-e28cd209c451 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:24.042497 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcf149e-fa40-4819-b8eb-e28cd209c451 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:24.043896 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 554/1124] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:24:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:24:24.049615 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6e2969-3f91-4a85-834d-2e3b69151e84 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:24.050029 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6e2969-3f91-4a85-834d-2e3b69151e84 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:24.050186 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6e2969-3f91-4a85-834d-2e3b69151e84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:24.067184 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-9b6e2969-3f91-4a85-834d-2e3b69151e84 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:24:24.068394 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-9b6e2969-3f91-4a85-834d-2e3b69151e84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:24:24.134309 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 571/1125] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:24:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 15:24:24.157216 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcf149e-fa40-4819-b8eb-e28cd209c451 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:24.261969 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcf149e-fa40-4819-b8eb-e28cd209c451 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:24.287700 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:24:24.297136 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ae8a2a-9ddf-4dd4-8c87-b9f642e74338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:24.297989 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ae8a2a-9ddf-4dd4-8c87-b9f642e74338 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:24.298276 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ae8a2a-9ddf-4dd4-8c87-b9f642e74338 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:24.298553 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ae8a2a-9ddf-4dd4-8c87-b9f642e74338 None admin] PATH_INFO: `/v3/roles/3627b4f45c8c4f41a7def6b15f1b8225` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:24.300071 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ae8a2a-9ddf-4dd4-8c87-b9f642e74338 None admin] RBAC: Authorizing `identity:get_role(role_id=3627b4f45c8c4f41a7def6b15f1b8225)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:24.301448 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ae8a2a-9ddf-4dd4-8c87-b9f642e74338 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:24.302826 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 555/1126] 10.222.0.99 () {56 vars in 1177 bytes} [Thu Jul 14 15:24:24 2022] GET /identity/v3/roles/3627b4f45c8c4f41a7def6b15f1b8225 => generated 223 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:24:24.308678 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6e2969-3f91-4a85-834d-2e3b69151e84 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:24.355071 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:24:24.411392 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6e2969-3f91-4a85-834d-2e3b69151e84 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:24.450251 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1cf6f1-e98f-401a-9b01-720e9daad79e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:24.451125 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1cf6f1-e98f-401a-9b01-720e9daad79e None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:24.451410 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1cf6f1-e98f-401a-9b01-720e9daad79e None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:24.451767 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1cf6f1-e98f-401a-9b01-720e9daad79e None admin] PATH_INFO: `/v3/users/672dba645ae244648d10e7a37e115b82` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:24.454302 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1cf6f1-e98f-401a-9b01-720e9daad79e None admin] RBAC: Authorizing `identity:get_user(user_id=672dba645ae244648d10e7a37e115b82)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:24.455800 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1cf6f1-e98f-401a-9b01-720e9daad79e None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:24.457470 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 572/1127] 10.222.0.99 () {56 vars in 1177 bytes} [Thu Jul 14 15:24:24 2022] GET /identity/v3/users/672dba645ae244648d10e7a37e115b82 => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:24:24.463249 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ae8a2a-9ddf-4dd4-8c87-b9f642e74338 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:24.563221 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ae8a2a-9ddf-4dd4-8c87-b9f642e74338 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:24.597073 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d77aecc-d4b6-4185-bee4-d6306a78791b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:24.597715 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d77aecc-d4b6-4185-bee4-d6306a78791b None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:24.598006 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d77aecc-d4b6-4185-bee4-d6306a78791b None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:24.598289 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d77aecc-d4b6-4185-bee4-d6306a78791b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:24.657468 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:24:24.665364 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d77aecc-d4b6-4185-bee4-d6306a78791b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:24.666823 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d77aecc-d4b6-4185-bee4-d6306a78791b None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:24.764478 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 556/1128] 10.222.0.99 () {58 vars in 1282 bytes} [Thu Jul 14 15:24:24 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 15:24:24.770719 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1cf6f1-e98f-401a-9b01-720e9daad79e None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:24.879327 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1cf6f1-e98f-401a-9b01-720e9daad79e None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:24.893648 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:24:24.914398 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed42f51-54d8-4fad-a9c9-2cecc0fcc30f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:24.915259 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed42f51-54d8-4fad-a9c9-2cecc0fcc30f None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:24.915550 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed42f51-54d8-4fad-a9c9-2cecc0fcc30f None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:24.915864 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed42f51-54d8-4fad-a9c9-2cecc0fcc30f None admin] PATH_INFO: `/v3/projects/1126c60f09584b14a95df33777f31900` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:24.917045 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed42f51-54d8-4fad-a9c9-2cecc0fcc30f None admin] RBAC: Authorizing `identity:get_project(project_id=1126c60f09584b14a95df33777f31900)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:24.918480 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed42f51-54d8-4fad-a9c9-2cecc0fcc30f None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:24.920202 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 573/1129] 10.222.0.99 () {56 vars in 1186 bytes} [Thu Jul 14 15:24:24 2022] GET /identity/v3/projects/1126c60f09584b14a95df33777f31900 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:24:24.927016 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d77aecc-d4b6-4185-bee4-d6306a78791b None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:25.018768 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:24:25.025652 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d77aecc-d4b6-4185-bee4-d6306a78791b None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:25.058869 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0413f8b8-093e-465f-b671-8fc804152de3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:25.059530 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0413f8b8-093e-465f-b671-8fc804152de3 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:25.059847 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0413f8b8-093e-465f-b671-8fc804152de3 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:25.060129 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0413f8b8-093e-465f-b671-8fc804152de3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:25.060960 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0413f8b8-093e-465f-b671-8fc804152de3 None admin] RBAC: Adding query filter params (role.id=3627b4f45c8c4f41a7def6b15f1b8225, scope.project.id=1126c60f09584b14a95df33777f31900, user.id=672dba645ae244648d10e7a37e115b82) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:24:25.061277 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0413f8b8-093e-465f-b671-8fc804152de3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:25.063450 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0413f8b8-093e-465f-b671-8fc804152de3 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:25.069255 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 557/1130] 10.222.0.99 () {56 vars in 1374 bytes} [Thu Jul 14 15:24:24 2022] GET /identity/v3/role_assignments?user.id=672dba645ae244648d10e7a37e115b82&scope.project.id=1126c60f09584b14a95df33777f31900&role.id=3627b4f45c8c4f41a7def6b15f1b8225 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:24:27.139546 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7654baf-e0a7-4e7d-9bc7-532f8a07c61e None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:27.140661 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7654baf-e0a7-4e7d-9bc7-532f8a07c61e None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:27.140902 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7654baf-e0a7-4e7d-9bc7-532f8a07c61e None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:27.142296 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 574/1131] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:24:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:24:27.147528 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb492a8b-4483-4fef-959d-3bec9f9b3faf None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:27.147951 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb492a8b-4483-4fef-959d-3bec9f9b3faf None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:27.148220 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb492a8b-4483-4fef-959d-3bec9f9b3faf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:27.165217 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-eb492a8b-4483-4fef-959d-3bec9f9b3faf None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:24:27.166415 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-eb492a8b-4483-4fef-959d-3bec9f9b3faf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:24:27.235765 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 558/1132] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:24:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 15:24:27.261211 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7654baf-e0a7-4e7d-9bc7-532f8a07c61e None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:27.366716 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7654baf-e0a7-4e7d-9bc7-532f8a07c61e None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:27.400989 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764e1a5f-4a8a-4c6e-8971-fd4f63047c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:27.401780 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764e1a5f-4a8a-4c6e-8971-fd4f63047c34 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:27.402068 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764e1a5f-4a8a-4c6e-8971-fd4f63047c34 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:27.402349 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764e1a5f-4a8a-4c6e-8971-fd4f63047c34 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:27.403518 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764e1a5f-4a8a-4c6e-8971-fd4f63047c34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:27.404990 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764e1a5f-4a8a-4c6e-8971-fd4f63047c34 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:27.406943 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 575/1133] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:24:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:24:27.412951 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb492a8b-4483-4fef-959d-3bec9f9b3faf None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:27.497571 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:24:27.510486 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb492a8b-4483-4fef-959d-3bec9f9b3faf None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:27.544270 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c79781-3486-49ed-8ea4-e75879055f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:27.544859 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c79781-3486-49ed-8ea4-e75879055f35 None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:27.545143 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c79781-3486-49ed-8ea4-e75879055f35 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:27.545421 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c79781-3486-49ed-8ea4-e75879055f35 None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:27.546161 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c79781-3486-49ed-8ea4-e75879055f35 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:27.547702 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c79781-3486-49ed-8ea4-e75879055f35 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:27.577922 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 559/1134] 10.222.0.99 () {60 vars in 1141 bytes} [Thu Jul 14 15:24:27 2022] POST /identity/v3/projects => generated 311 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:24:29.665488 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c5c687-238c-4690-a05c-4beb29865e52 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:29.666045 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c5c687-238c-4690-a05c-4beb29865e52 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:29.666045 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c5c687-238c-4690-a05c-4beb29865e52 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:29.667406 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 576/1135] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:24:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:24:29.672491 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69795d40-8b49-4b37-8bb2-93a996249795 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:29.672895 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69795d40-8b49-4b37-8bb2-93a996249795 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:29.673057 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69795d40-8b49-4b37-8bb2-93a996249795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:29.690666 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-69795d40-8b49-4b37-8bb2-93a996249795 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:24:29.691912 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-69795d40-8b49-4b37-8bb2-93a996249795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:24:29.757975 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 560/1136] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:24:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 15:24:29.783815 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c5c687-238c-4690-a05c-4beb29865e52 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:29.834310 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:24:29.883801 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c5c687-238c-4690-a05c-4beb29865e52 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:29.917502 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c44a7c-eed4-4d0e-939c-ab59ad42bfba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:29.918228 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c44a7c-eed4-4d0e-939c-ab59ad42bfba None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:29.918510 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c44a7c-eed4-4d0e-939c-ab59ad42bfba None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:29.918782 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c44a7c-eed4-4d0e-939c-ab59ad42bfba None admin] PATH_INFO: `/v3/domains/default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:29.919966 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c44a7c-eed4-4d0e-939c-ab59ad42bfba None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:29.922031 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c44a7c-eed4-4d0e-939c-ab59ad42bfba None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:29.923651 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 577/1137] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:24:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:24:29.929590 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69795d40-8b49-4b37-8bb2-93a996249795 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:29.997587 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:24:30.036917 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69795d40-8b49-4b37-8bb2-93a996249795 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:30.069920 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbfd0d3-865f-48bb-8515-4299450d7fc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:30.070581 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbfd0d3-865f-48bb-8515-4299450d7fc7 None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:30.070866 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbfd0d3-865f-48bb-8515-4299450d7fc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:30.071144 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbfd0d3-865f-48bb-8515-4299450d7fc7 None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:30.071875 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbfd0d3-865f-48bb-8515-4299450d7fc7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:30.073301 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbfd0d3-865f-48bb-8515-4299450d7fc7 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:30.098490 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 561/1138] 10.222.0.99 () {60 vars in 1132 bytes} [Thu Jul 14 15:24:29 2022] POST /identity/v3/users => generated 286 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:24:32.157884 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d33f8a9-4c5a-4151-a543-f789ffec8734 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:32.158438 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d33f8a9-4c5a-4151-a543-f789ffec8734 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:32.158438 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d33f8a9-4c5a-4151-a543-f789ffec8734 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:32.159819 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 578/1139] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:24:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:24:32.164967 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0943322-c210-4e71-a02a-57352e05089f None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:32.165392 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0943322-c210-4e71-a02a-57352e05089f None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:32.165531 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0943322-c210-4e71-a02a-57352e05089f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:32.183484 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-f0943322-c210-4e71-a02a-57352e05089f None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:24:32.184708 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f0943322-c210-4e71-a02a-57352e05089f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:24:32.251440 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 562/1140] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:24:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 15:24:32.275608 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d33f8a9-4c5a-4151-a543-f789ffec8734 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:32.374868 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d33f8a9-4c5a-4151-a543-f789ffec8734 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:32.392232 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:24:32.410164 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c749bf3-d836-4a74-8e44-3949750c1aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:32.410957 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c749bf3-d836-4a74-8e44-3949750c1aa7 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:32.411241 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c749bf3-d836-4a74-8e44-3949750c1aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:32.411517 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c749bf3-d836-4a74-8e44-3949750c1aa7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:32.421477 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c749bf3-d836-4a74-8e44-3949750c1aa7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:24:32.428996 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-5c749bf3-d836-4a74-8e44-3949750c1aa7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:24:32.428996 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:24:32.428996 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:24:32.428996 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:24:32.428996 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:24:32.428996 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:24:32.428996 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:24:32.428996 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:24:32.428996 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:24:32.428996 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:24:32.428996 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:24:32.428996 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:24:32.428996 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:24:32.428996 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:24:32.428996 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:24:32.428996 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:24:32.430074 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:24:32.430074 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:24:32.430074 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:24:32.430074 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:24:32.430074 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:24:32.430074 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:24:32.430074 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:24:32.430074 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:24:32.430074 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:24:32.430074 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:24:32.430074 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:24:32.430074 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:24:32.430074 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:24:32.430074 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:24:32.430074 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:24:32.430074 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:24:32.430929 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:24:32.430929 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:24:32.430929 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:24:32.430929 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:24:32.430929 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:24:32.430929 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:24:32.430929 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:24:32.430929 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:24:32.430929 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:24:32.430929 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:24:32.430929 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:24:32.430929 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:24:32.430929 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 579/1141] 10.222.0.99 () {56 vars in 1096 bytes} [Thu Jul 14 15:24:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:24:32.436237 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0943322-c210-4e71-a02a-57352e05089f None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:32.490482 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:24:32.533915 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0943322-c210-4e71-a02a-57352e05089f None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:32.567276 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155875d4-5aa5-4f93-9403-785effa3533e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:32.567980 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155875d4-5aa5-4f93-9403-785effa3533e None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:32.568268 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155875d4-5aa5-4f93-9403-785effa3533e None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:32.568564 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155875d4-5aa5-4f93-9403-785effa3533e None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:32.569301 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155875d4-5aa5-4f93-9403-785effa3533e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:24:32.569614 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155875d4-5aa5-4f93-9403-785effa3533e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:32.570983 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155875d4-5aa5-4f93-9403-785effa3533e None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:32.580517 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 563/1142] 10.222.0.99 () {56 vars in 1099 bytes} [Thu Jul 14 15:24:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:24:32.586319 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c749bf3-d836-4a74-8e44-3949750c1aa7 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:32.685747 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c749bf3-d836-4a74-8e44-3949750c1aa7 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:32.720265 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fa0840-d1df-4adf-821a-b96193a6d96b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:32.721131 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fa0840-d1df-4adf-821a-b96193a6d96b None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:32.721418 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fa0840-d1df-4adf-821a-b96193a6d96b None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:32.721698 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fa0840-d1df-4adf-821a-b96193a6d96b None admin] PATH_INFO: `/v3/users/2447cca9c4dd425e890750879cd54802` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:32.734848 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fa0840-d1df-4adf-821a-b96193a6d96b None admin] RBAC: Authorizing `identity:get_user(user_id=2447cca9c4dd425e890750879cd54802)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:32.736326 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fa0840-d1df-4adf-821a-b96193a6d96b None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:32.737971 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 580/1143] 10.222.0.99 () {56 vars in 1177 bytes} [Thu Jul 14 15:24:32 2022] GET /identity/v3/users/2447cca9c4dd425e890750879cd54802 => generated 286 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:24:32.743744 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155875d4-5aa5-4f93-9403-785effa3533e None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:32.841341 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155875d4-5aa5-4f93-9403-785effa3533e None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:32.857583 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:24:32.874715 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8697178a-2c1d-4927-8f9f-de12226787ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:32.875362 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8697178a-2c1d-4927-8f9f-de12226787ff None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:32.875668 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8697178a-2c1d-4927-8f9f-de12226787ff None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:32.875947 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8697178a-2c1d-4927-8f9f-de12226787ff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:32.945274 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8697178a-2c1d-4927-8f9f-de12226787ff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:32.946772 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8697178a-2c1d-4927-8f9f-de12226787ff None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:33.043207 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 564/1144] 10.222.0.99 () {58 vars in 1282 bytes} [Thu Jul 14 15:24:32 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 15:24:33.050692 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fa0840-d1df-4adf-821a-b96193a6d96b None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:33.086506 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:24:33.150080 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fa0840-d1df-4adf-821a-b96193a6d96b None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:33.183094 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66d395b-b019-4c14-93ac-aa0c6ac8646d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:33.183971 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66d395b-b019-4c14-93ac-aa0c6ac8646d None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:33.184253 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66d395b-b019-4c14-93ac-aa0c6ac8646d None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:33.184532 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66d395b-b019-4c14-93ac-aa0c6ac8646d None admin] PATH_INFO: `/v3/projects/9e623fbe3d94469bb0d4f9c8a0a357ed` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:33.185689 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66d395b-b019-4c14-93ac-aa0c6ac8646d None admin] RBAC: Authorizing `identity:get_project(project_id=9e623fbe3d94469bb0d4f9c8a0a357ed)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:33.187743 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66d395b-b019-4c14-93ac-aa0c6ac8646d None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:33.189407 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 581/1145] 10.222.0.99 () {56 vars in 1186 bytes} [Thu Jul 14 15:24:33 2022] GET /identity/v3/projects/9e623fbe3d94469bb0d4f9c8a0a357ed => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:24:33.195555 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8697178a-2c1d-4927-8f9f-de12226787ff None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:33.214574 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:24:33.294172 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8697178a-2c1d-4927-8f9f-de12226787ff None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:33.327928 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5454f58-7ab6-4e4e-baa9-1ac956a5f848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:33.328568 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5454f58-7ab6-4e4e-baa9-1ac956a5f848 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:33.328856 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5454f58-7ab6-4e4e-baa9-1ac956a5f848 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:33.329135 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5454f58-7ab6-4e4e-baa9-1ac956a5f848 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:33.329956 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5454f58-7ab6-4e4e-baa9-1ac956a5f848 None admin] RBAC: Adding query filter params (role.id=d0f6ba1314f14d9f9c816bdb5d0c3aac, scope.project.id=9e623fbe3d94469bb0d4f9c8a0a357ed, user.id=2447cca9c4dd425e890750879cd54802) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:24:33.330271 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5454f58-7ab6-4e4e-baa9-1ac956a5f848 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:33.331722 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5454f58-7ab6-4e4e-baa9-1ac956a5f848 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:33.338778 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 565/1146] 10.222.0.99 () {56 vars in 1374 bytes} [Thu Jul 14 15:24:33 2022] GET /identity/v3/role_assignments?user.id=2447cca9c4dd425e890750879cd54802&scope.project.id=9e623fbe3d94469bb0d4f9c8a0a357ed&role.id=d0f6ba1314f14d9f9c816bdb5d0c3aac => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:24:35.357855 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c44c82a-0684-4a84-b942-f53a372dc091 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:35.358362 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c44c82a-0684-4a84-b942-f53a372dc091 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:35.358362 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c44c82a-0684-4a84-b942-f53a372dc091 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:35.359868 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 582/1147] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:24:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:24:35.365183 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11a7889-d6f1-4d95-8ed1-e558b20c20fe None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:35.365599 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11a7889-d6f1-4d95-8ed1-e558b20c20fe None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:35.365742 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11a7889-d6f1-4d95-8ed1-e558b20c20fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:35.381969 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-e11a7889-d6f1-4d95-8ed1-e558b20c20fe None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:24:35.383138 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e11a7889-d6f1-4d95-8ed1-e558b20c20fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:24:35.449983 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 566/1148] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:24:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 15:24:35.473311 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c44c82a-0684-4a84-b942-f53a372dc091 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:35.574381 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c44c82a-0684-4a84-b942-f53a372dc091 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:35.576675 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:24:35.611889 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6236a67-e010-43b8-8d56-b0ecb3aab4bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:35.612613 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6236a67-e010-43b8-8d56-b0ecb3aab4bf None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:35.612895 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6236a67-e010-43b8-8d56-b0ecb3aab4bf None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:35.613173 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6236a67-e010-43b8-8d56-b0ecb3aab4bf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:35.623106 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6236a67-e010-43b8-8d56-b0ecb3aab4bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:24:35.631478 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-b6236a67-e010-43b8-8d56-b0ecb3aab4bf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:24:35.631478 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:24:35.631478 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:24:35.631478 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:24:35.631478 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:24:35.631478 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:24:35.631478 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:24:35.631478 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:24:35.631478 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:24:35.631478 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:24:35.631478 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:24:35.631478 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:24:35.631478 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:24:35.631478 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:24:35.631478 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:24:35.631478 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:24:35.632612 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:24:35.632612 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:24:35.632612 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:24:35.632612 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:24:35.632612 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:24:35.632612 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:24:35.632612 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:24:35.632612 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:24:35.632612 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:24:35.632612 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:24:35.632612 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:24:35.632612 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:24:35.632612 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:24:35.632612 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:24:35.632612 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:24:35.632612 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:24:35.633510 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:24:35.633510 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:24:35.633510 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:24:35.633510 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:24:35.633510 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:24:35.633510 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:24:35.633510 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:24:35.633510 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:24:35.633510 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:24:35.633510 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:24:35.633510 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:24:35.633510 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:24:35.633510 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 583/1149] 10.222.0.99 () {56 vars in 1096 bytes} [Thu Jul 14 15:24:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:24:35.639200 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11a7889-d6f1-4d95-8ed1-e558b20c20fe None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:35.653660 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:24:35.735933 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11a7889-d6f1-4d95-8ed1-e558b20c20fe None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:35.769876 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3007386-7e80-42ac-bbe2-26a951409941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:35.770536 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3007386-7e80-42ac-bbe2-26a951409941 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:35.770821 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3007386-7e80-42ac-bbe2-26a951409941 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:35.771097 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3007386-7e80-42ac-bbe2-26a951409941 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:35.771869 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3007386-7e80-42ac-bbe2-26a951409941 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:24:35.772177 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3007386-7e80-42ac-bbe2-26a951409941 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:35.773539 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3007386-7e80-42ac-bbe2-26a951409941 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:35.781412 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 567/1150] 10.222.0.99 () {56 vars in 1099 bytes} [Thu Jul 14 15:24:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:24:35.787588 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6236a67-e010-43b8-8d56-b0ecb3aab4bf None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:35.885499 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6236a67-e010-43b8-8d56-b0ecb3aab4bf None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:35.918841 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc429971-a1e1-4318-b9e7-29280033410d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:35.919729 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc429971-a1e1-4318-b9e7-29280033410d None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:35.920023 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc429971-a1e1-4318-b9e7-29280033410d None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:35.920307 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc429971-a1e1-4318-b9e7-29280033410d None admin] PATH_INFO: `/v3/users/2447cca9c4dd425e890750879cd54802` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:35.922029 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc429971-a1e1-4318-b9e7-29280033410d None admin] RBAC: Authorizing `identity:get_user(user_id=2447cca9c4dd425e890750879cd54802)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:35.923463 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc429971-a1e1-4318-b9e7-29280033410d None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:35.925116 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 584/1151] 10.222.0.99 () {56 vars in 1177 bytes} [Thu Jul 14 15:24:35 2022] GET /identity/v3/users/2447cca9c4dd425e890750879cd54802 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:24:35.930993 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3007386-7e80-42ac-bbe2-26a951409941 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:36.006392 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:24:36.031466 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3007386-7e80-42ac-bbe2-26a951409941 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:36.066291 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99c5326-61b4-4c0f-b2af-4ed87c4e5a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:36.067154 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99c5326-61b4-4c0f-b2af-4ed87c4e5a79 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:36.067446 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99c5326-61b4-4c0f-b2af-4ed87c4e5a79 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:36.067756 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99c5326-61b4-4c0f-b2af-4ed87c4e5a79 None admin] PATH_INFO: `/v3/projects/9e623fbe3d94469bb0d4f9c8a0a357ed` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:36.068952 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99c5326-61b4-4c0f-b2af-4ed87c4e5a79 None admin] RBAC: Authorizing `identity:get_project(project_id=9e623fbe3d94469bb0d4f9c8a0a357ed)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:36.070387 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99c5326-61b4-4c0f-b2af-4ed87c4e5a79 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:36.072120 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 568/1152] 10.222.0.99 () {56 vars in 1186 bytes} [Thu Jul 14 15:24:35 2022] GET /identity/v3/projects/9e623fbe3d94469bb0d4f9c8a0a357ed => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:24:36.078251 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc429971-a1e1-4318-b9e7-29280033410d None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:36.109614 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:24:36.176715 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc429971-a1e1-4318-b9e7-29280033410d None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:36.209755 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e364ed3a-7d25-4e63-8358-8ed77f605fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:36.210633 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e364ed3a-7d25-4e63-8358-8ed77f605fce None admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:36.210916 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e364ed3a-7d25-4e63-8358-8ed77f605fce None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:36.211189 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e364ed3a-7d25-4e63-8358-8ed77f605fce None admin] PATH_INFO: `/v3/projects/9e623fbe3d94469bb0d4f9c8a0a357ed/users/2447cca9c4dd425e890750879cd54802/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:36.213141 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e364ed3a-7d25-4e63-8358-8ed77f605fce None admin] RBAC: Authorizing `identity:create_grant(project_id=9e623fbe3d94469bb0d4f9c8a0a357ed, user_id=2447cca9c4dd425e890750879cd54802, role_id=d0f6ba1314f14d9f9c816bdb5d0c3aac)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:36.214557 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e364ed3a-7d25-4e63-8358-8ed77f605fce None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:36.227215 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 585/1153] 10.222.0.99 () {58 vars in 1439 bytes} [Thu Jul 14 15:24:36 2022] PUT /identity/v3/projects/9e623fbe3d94469bb0d4f9c8a0a357ed/users/2447cca9c4dd425e890750879cd54802/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:24:38.270339 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f36e48f-650f-4459-909f-e4f48334354f None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:38.270842 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f36e48f-650f-4459-909f-e4f48334354f None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:38.270842 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f36e48f-650f-4459-909f-e4f48334354f None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:38.272240 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 569/1154] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:24:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:24:38.277664 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccdee12-ff02-4f99-b558-244119e208be None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:38.278111 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccdee12-ff02-4f99-b558-244119e208be None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:38.278237 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccdee12-ff02-4f99-b558-244119e208be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:38.295343 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-dccdee12-ff02-4f99-b558-244119e208be None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:24:38.297225 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-dccdee12-ff02-4f99-b558-244119e208be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:24:38.373793 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 586/1155] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:24:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 15:24:38.400428 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f36e48f-650f-4459-909f-e4f48334354f None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:38.504910 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f36e48f-650f-4459-909f-e4f48334354f None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:38.538830 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87099ff8-804d-4fa8-9a6c-fa9581ac7c77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:38.539544 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87099ff8-804d-4fa8-9a6c-fa9581ac7c77 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:38.539862 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87099ff8-804d-4fa8-9a6c-fa9581ac7c77 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:38.540147 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87099ff8-804d-4fa8-9a6c-fa9581ac7c77 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:38.550697 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87099ff8-804d-4fa8-9a6c-fa9581ac7c77 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:24:38.558413 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-87099ff8-804d-4fa8-9a6c-fa9581ac7c77 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:24:38.558413 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:24:38.558413 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:24:38.558413 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:24:38.558413 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:24:38.558413 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:24:38.558413 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:24:38.558413 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:24:38.558413 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:24:38.558413 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:24:38.558413 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:24:38.558413 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:24:38.558413 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:24:38.558413 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:24:38.558413 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:24:38.558413 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:24:38.559501 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:24:38.559501 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:24:38.559501 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:24:38.559501 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:24:38.559501 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:24:38.559501 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:24:38.559501 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:24:38.559501 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:24:38.559501 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:24:38.559501 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:24:38.559501 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:24:38.559501 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:24:38.559501 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:24:38.559501 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:24:38.559501 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:24:38.559501 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:24:38.560434 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:24:38.560434 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:24:38.560434 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:24:38.560434 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:24:38.560434 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:24:38.560434 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:24:38.560434 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:24:38.560434 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:24:38.560434 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:24:38.560434 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:24:38.560434 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:24:38.560434 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:24:38.560434 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 570/1156] 10.222.0.99 () {56 vars in 1096 bytes} [Thu Jul 14 15:24:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:24:38.566372 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccdee12-ff02-4f99-b558-244119e208be None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:38.590031 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:24:38.675721 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccdee12-ff02-4f99-b558-244119e208be None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:38.711332 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da5ced7-292d-4f7c-81c0-6b9f3e076ff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:38.712037 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da5ced7-292d-4f7c-81c0-6b9f3e076ff5 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:38.712327 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da5ced7-292d-4f7c-81c0-6b9f3e076ff5 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:38.712611 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da5ced7-292d-4f7c-81c0-6b9f3e076ff5 None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:38.713371 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da5ced7-292d-4f7c-81c0-6b9f3e076ff5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:24:38.713683 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da5ced7-292d-4f7c-81c0-6b9f3e076ff5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:38.715078 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da5ced7-292d-4f7c-81c0-6b9f3e076ff5 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:38.724151 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 587/1157] 10.222.0.99 () {56 vars in 1099 bytes} [Thu Jul 14 15:24:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:24:38.730172 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87099ff8-804d-4fa8-9a6c-fa9581ac7c77 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:38.733153 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:24:38.837906 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87099ff8-804d-4fa8-9a6c-fa9581ac7c77 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:38.874948 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf153e4-f2f3-4fd0-90a4-29f1882f76fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:38.875854 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf153e4-f2f3-4fd0-90a4-29f1882f76fc None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:38.876144 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf153e4-f2f3-4fd0-90a4-29f1882f76fc None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:38.876423 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf153e4-f2f3-4fd0-90a4-29f1882f76fc None admin] PATH_INFO: `/v3/users/2447cca9c4dd425e890750879cd54802` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:38.877647 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf153e4-f2f3-4fd0-90a4-29f1882f76fc None admin] RBAC: Authorizing `identity:get_user(user_id=2447cca9c4dd425e890750879cd54802)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:38.879091 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf153e4-f2f3-4fd0-90a4-29f1882f76fc None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:38.880781 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 571/1158] 10.222.0.99 () {56 vars in 1177 bytes} [Thu Jul 14 15:24:38 2022] GET /identity/v3/users/2447cca9c4dd425e890750879cd54802 => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:24:38.886796 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da5ced7-292d-4f7c-81c0-6b9f3e076ff5 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:38.977313 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:24:38.994114 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da5ced7-292d-4f7c-81c0-6b9f3e076ff5 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:39.027869 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c6dcdf-9df0-4a32-a251-ed3d0851f302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:39.028522 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c6dcdf-9df0-4a32-a251-ed3d0851f302 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:39.028813 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c6dcdf-9df0-4a32-a251-ed3d0851f302 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:39.029094 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c6dcdf-9df0-4a32-a251-ed3d0851f302 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:39.095972 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c6dcdf-9df0-4a32-a251-ed3d0851f302 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:39.097348 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c6dcdf-9df0-4a32-a251-ed3d0851f302 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:39.177972 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:24:39.194774 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 588/1159] 10.222.0.99 () {58 vars in 1282 bytes} [Thu Jul 14 15:24:38 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 15:24:39.200708 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf153e4-f2f3-4fd0-90a4-29f1882f76fc None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:39.271058 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:24:39.302381 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf153e4-f2f3-4fd0-90a4-29f1882f76fc None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:39.336067 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48428e93-b1ff-4bcb-89e1-ddefef7a3940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:39.336924 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48428e93-b1ff-4bcb-89e1-ddefef7a3940 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:39.337210 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48428e93-b1ff-4bcb-89e1-ddefef7a3940 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:39.337488 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48428e93-b1ff-4bcb-89e1-ddefef7a3940 None admin] PATH_INFO: `/v3/projects/9e623fbe3d94469bb0d4f9c8a0a357ed` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:39.338668 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48428e93-b1ff-4bcb-89e1-ddefef7a3940 None admin] RBAC: Authorizing `identity:get_project(project_id=9e623fbe3d94469bb0d4f9c8a0a357ed)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:39.340137 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48428e93-b1ff-4bcb-89e1-ddefef7a3940 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:39.341848 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 572/1160] 10.222.0.99 () {56 vars in 1186 bytes} [Thu Jul 14 15:24:39 2022] GET /identity/v3/projects/9e623fbe3d94469bb0d4f9c8a0a357ed => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:24:39.348660 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c6dcdf-9df0-4a32-a251-ed3d0851f302 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:39.446445 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c6dcdf-9df0-4a32-a251-ed3d0851f302 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:39.479865 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2746742d-2174-4052-a560-907d706bee77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:39.480517 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2746742d-2174-4052-a560-907d706bee77 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:39.480808 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2746742d-2174-4052-a560-907d706bee77 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:39.481091 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2746742d-2174-4052-a560-907d706bee77 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:39.481931 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2746742d-2174-4052-a560-907d706bee77 None admin] RBAC: Adding query filter params (role.id=d0f6ba1314f14d9f9c816bdb5d0c3aac, scope.project.id=9e623fbe3d94469bb0d4f9c8a0a357ed, user.id=2447cca9c4dd425e890750879cd54802) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:24:39.482247 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2746742d-2174-4052-a560-907d706bee77 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:39.484475 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2746742d-2174-4052-a560-907d706bee77 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:39.490179 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 589/1161] 10.222.0.99 () {56 vars in 1374 bytes} [Thu Jul 14 15:24:39 2022] GET /identity/v3/role_assignments?user.id=2447cca9c4dd425e890750879cd54802&scope.project.id=9e623fbe3d94469bb0d4f9c8a0a357ed&role.id=d0f6ba1314f14d9f9c816bdb5d0c3aac => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:24:41.540933 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc43720-c1ca-4a4d-b752-a54d2bc76146 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:41.541522 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc43720-c1ca-4a4d-b752-a54d2bc76146 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:41.541522 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc43720-c1ca-4a4d-b752-a54d2bc76146 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:41.542872 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 573/1162] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:24:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:24:41.548512 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6b7daf-e005-4bb7-a436-9f871b757f66 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:41.548941 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6b7daf-e005-4bb7-a436-9f871b757f66 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:41.549088 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6b7daf-e005-4bb7-a436-9f871b757f66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:41.566103 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-db6b7daf-e005-4bb7-a436-9f871b757f66 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:24:41.567365 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-db6b7daf-e005-4bb7-a436-9f871b757f66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:24:41.609985 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:24:41.633681 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 590/1163] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:24:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 15:24:41.657125 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc43720-c1ca-4a4d-b752-a54d2bc76146 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:41.758054 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc43720-c1ca-4a4d-b752-a54d2bc76146 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:41.790754 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f5d15e-0ed4-4e15-ba2b-f046bbb42b8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:41.791438 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f5d15e-0ed4-4e15-ba2b-f046bbb42b8e None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:41.791699 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f5d15e-0ed4-4e15-ba2b-f046bbb42b8e None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:41.791979 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f5d15e-0ed4-4e15-ba2b-f046bbb42b8e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:41.799737 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:24:41.857720 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f5d15e-0ed4-4e15-ba2b-f046bbb42b8e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:41.859178 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f5d15e-0ed4-4e15-ba2b-f046bbb42b8e None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:41.957243 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 574/1164] 10.222.0.99 () {58 vars in 1282 bytes} [Thu Jul 14 15:24:41 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 15:24:41.963287 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6b7daf-e005-4bb7-a436-9f871b757f66 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:42.066633 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6b7daf-e005-4bb7-a436-9f871b757f66 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:42.101945 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2326337d-5094-42f6-927b-f98e4544d185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:42.102765 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2326337d-5094-42f6-927b-f98e4544d185 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:42.103054 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2326337d-5094-42f6-927b-f98e4544d185 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:42.103338 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2326337d-5094-42f6-927b-f98e4544d185 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:42.110787 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2326337d-5094-42f6-927b-f98e4544d185 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:42.112263 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2326337d-5094-42f6-927b-f98e4544d185 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:42.127222 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-2326337d-5094-42f6-927b-f98e4544d185 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 14 15:24:42.127222 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:24:42.127222 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 14 15:24:42.127222 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 14 15:24:42.127222 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 15:24:42.127222 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 15:24:42.127222 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 15:24:42.127222 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 15:24:42.127222 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 14 15:24:42.127222 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:24:42.127222 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 15:24:42.127222 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:24:42.127222 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:24:42.127222 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:24:42.127222 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:24:42.127222 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:24:42.127222 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:24:42.128382 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:24:42.128382 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:24:42.128382 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:24:42.128382 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:24:42.128382 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:24:42.128382 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:24:42.128382 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:24:42.128382 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:24:42.128382 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:24:42.128382 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:24:42.128382 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:24:42.128382 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:24:42.128382 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:24:42.128382 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:24:42.128382 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:24:42.128382 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:24:42.129455 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:24:42.129455 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:24:42.129455 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:24:42.129455 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:24:42.129455 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:24:42.129455 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:24:42.129455 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:24:42.129455 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:24:42.129455 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:24:42.129455 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:24:42.129455 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 14 15:24:42.129455 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:24:42.129455 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 14 15:24:42.129455 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:24:42.129455 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 591/1165] 10.222.0.99 () {56 vars in 1117 bytes} [Thu Jul 14 15:24:41 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:24:42.134533 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f5d15e-0ed4-4e15-ba2b-f046bbb42b8e None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:42.173198 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:24:42.231493 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f5d15e-0ed4-4e15-ba2b-f046bbb42b8e None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:42.266248 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc5bda2-0ba8-416f-80e6-cd1e68ce21de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:42.266918 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc5bda2-0ba8-416f-80e6-cd1e68ce21de None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:42.267209 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc5bda2-0ba8-416f-80e6-cd1e68ce21de None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:42.267491 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc5bda2-0ba8-416f-80e6-cd1e68ce21de None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:42.268249 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc5bda2-0ba8-416f-80e6-cd1e68ce21de None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:24:42.268560 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc5bda2-0ba8-416f-80e6-cd1e68ce21de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:42.269926 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc5bda2-0ba8-416f-80e6-cd1e68ce21de None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:42.277195 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 575/1166] 10.222.0.99 () {56 vars in 1115 bytes} [Thu Jul 14 15:24:42 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:24:44.344440 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfe5c99-503a-4ad8-a266-f2fc642b395c None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:44.344965 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfe5c99-503a-4ad8-a266-f2fc642b395c None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:44.344965 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfe5c99-503a-4ad8-a266-f2fc642b395c None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:44.346335 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 592/1167] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:24:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:24:44.351543 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ab7676-3778-418f-a533-d0df033eb93d None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:44.351978 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ab7676-3778-418f-a533-d0df033eb93d None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:44.352145 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ab7676-3778-418f-a533-d0df033eb93d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:44.368809 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-04ab7676-3778-418f-a533-d0df033eb93d None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:24:44.370009 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-04ab7676-3778-418f-a533-d0df033eb93d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:24:44.435486 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 576/1168] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:24:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 15:24:44.458280 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfe5c99-503a-4ad8-a266-f2fc642b395c None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:44.499946 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:24:44.563821 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfe5c99-503a-4ad8-a266-f2fc642b395c None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:44.596727 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea39217a-1fa4-4966-9667-10811ae28b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:44.597319 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea39217a-1fa4-4966-9667-10811ae28b78 None admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:44.597610 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea39217a-1fa4-4966-9667-10811ae28b78 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:44.597893 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea39217a-1fa4-4966-9667-10811ae28b78 None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:44.598646 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea39217a-1fa4-4966-9667-10811ae28b78 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:44.600055 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea39217a-1fa4-4966-9667-10811ae28b78 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:44.614844 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 593/1169] 10.222.0.99 () {60 vars in 1138 bytes} [Thu Jul 14 15:24:44 2022] POST /identity/v3/domains => generated 267 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:24:46.678633 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bf622b-96e0-49b2-a0ea-9683164569a2 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:46.679172 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bf622b-96e0-49b2-a0ea-9683164569a2 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:46.679172 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bf622b-96e0-49b2-a0ea-9683164569a2 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:46.680907 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 577/1170] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:24:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:24:46.686083 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99d2b9a-a672-4db5-bfb1-fa431605f934 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:46.686528 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99d2b9a-a672-4db5-bfb1-fa431605f934 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:46.686660 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99d2b9a-a672-4db5-bfb1-fa431605f934 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:46.704448 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-d99d2b9a-a672-4db5-bfb1-fa431605f934 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:24:46.705659 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-d99d2b9a-a672-4db5-bfb1-fa431605f934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:24:46.769833 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 594/1171] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:24:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 15:24:46.793193 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bf622b-96e0-49b2-a0ea-9683164569a2 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:46.825466 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:24:46.896790 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bf622b-96e0-49b2-a0ea-9683164569a2 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:46.929994 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa351aa-b205-46a0-a8cf-47e3a7841235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:46.930784 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa351aa-b205-46a0-a8cf-47e3a7841235 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:46.931070 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa351aa-b205-46a0-a8cf-47e3a7841235 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:46.931350 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa351aa-b205-46a0-a8cf-47e3a7841235 None admin] PATH_INFO: `/v3/domains/d4979fc44f394dd7a953a5be080d3dac` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:46.941582 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa351aa-b205-46a0-a8cf-47e3a7841235 None admin] RBAC: Authorizing `identity:get_domain(domain_id=d4979fc44f394dd7a953a5be080d3dac)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:46.943227 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa351aa-b205-46a0-a8cf-47e3a7841235 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:46.944904 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 578/1172] 10.222.0.99 () {56 vars in 1183 bytes} [Thu Jul 14 15:24:46 2022] GET /identity/v3/domains/d4979fc44f394dd7a953a5be080d3dac => generated 267 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:24:46.950908 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99d2b9a-a672-4db5-bfb1-fa431605f934 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:46.978532 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:24:47.055037 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99d2b9a-a672-4db5-bfb1-fa431605f934 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:47.089742 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8cca95-bb91-4d06-8669-3da35c996f1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:47.090407 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8cca95-bb91-4d06-8669-3da35c996f1e None admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:47.090694 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8cca95-bb91-4d06-8669-3da35c996f1e None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:47.090972 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8cca95-bb91-4d06-8669-3da35c996f1e None admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:47.091724 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8cca95-bb91-4d06-8669-3da35c996f1e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:47.093616 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8cca95-bb91-4d06-8669-3da35c996f1e None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:47.118186 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 595/1173] 10.222.0.99 () {60 vars in 1141 bytes} [Thu Jul 14 15:24:46 2022] POST /identity/v3/projects => generated 361 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 15:24:49.249404 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a151798e-ab96-4475-9e54-692efff19110 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:49.249933 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a151798e-ab96-4475-9e54-692efff19110 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:49.249933 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a151798e-ab96-4475-9e54-692efff19110 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:49.251255 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 579/1174] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:24:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:24:49.256563 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161d597a-dad4-4d15-95ee-69fa594d4edb None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:49.256954 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161d597a-dad4-4d15-95ee-69fa594d4edb None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:49.257132 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161d597a-dad4-4d15-95ee-69fa594d4edb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:49.273894 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-161d597a-dad4-4d15-95ee-69fa594d4edb None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:24:49.275099 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-161d597a-dad4-4d15-95ee-69fa594d4edb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:24:49.341504 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 596/1175] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:24:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 15:24:49.366963 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a151798e-ab96-4475-9e54-692efff19110 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:49.460170 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:24:49.471343 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a151798e-ab96-4475-9e54-692efff19110 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:49.505279 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028e2c42-3eb5-4864-b2a6-afabefabff90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:49.506055 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028e2c42-3eb5-4864-b2a6-afabefabff90 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:49.506337 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028e2c42-3eb5-4864-b2a6-afabefabff90 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:49.506611 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028e2c42-3eb5-4864-b2a6-afabefabff90 None admin] PATH_INFO: `/v3/domains/d4979fc44f394dd7a953a5be080d3dac` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:49.507808 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028e2c42-3eb5-4864-b2a6-afabefabff90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=d4979fc44f394dd7a953a5be080d3dac)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:49.509244 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028e2c42-3eb5-4864-b2a6-afabefabff90 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:49.510860 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 580/1176] 10.222.0.99 () {56 vars in 1183 bytes} [Thu Jul 14 15:24:49 2022] GET /identity/v3/domains/d4979fc44f394dd7a953a5be080d3dac => generated 267 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:24:49.517499 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161d597a-dad4-4d15-95ee-69fa594d4edb None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:49.529044 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:24:49.615388 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161d597a-dad4-4d15-95ee-69fa594d4edb None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:49.648363 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b237ca0-5466-47c3-9f2f-ed539f2ef67d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:49.649024 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b237ca0-5466-47c3-9f2f-ed539f2ef67d None admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:49.649311 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b237ca0-5466-47c3-9f2f-ed539f2ef67d None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:49.649588 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b237ca0-5466-47c3-9f2f-ed539f2ef67d None admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:49.650299 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b237ca0-5466-47c3-9f2f-ed539f2ef67d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:49.651758 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b237ca0-5466-47c3-9f2f-ed539f2ef67d None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:49.671549 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 597/1177] 10.222.0.99 () {60 vars in 1132 bytes} [Thu Jul 14 15:24:49 2022] POST /identity/v3/users => generated 311 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:24:51.814205 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41994887-64e5-42f0-8c8c-301c35050aef None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:51.814746 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41994887-64e5-42f0-8c8c-301c35050aef None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:51.814746 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41994887-64e5-42f0-8c8c-301c35050aef None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:51.816170 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 581/1178] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:24:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:24:51.821513 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ad914a-b609-458b-9bd5-f1e56591b796 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:51.821956 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ad914a-b609-458b-9bd5-f1e56591b796 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:51.822082 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ad914a-b609-458b-9bd5-f1e56591b796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:51.839037 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-a9ad914a-b609-458b-9bd5-f1e56591b796 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:24:51.840258 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-a9ad914a-b609-458b-9bd5-f1e56591b796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:24:51.899273 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:24:51.908018 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 598/1179] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:24:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 15:24:51.930905 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41994887-64e5-42f0-8c8c-301c35050aef None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:52.032609 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41994887-64e5-42f0-8c8c-301c35050aef None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:52.065556 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d0dcfe-505a-47f5-8cba-44bfa3446dc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:52.066258 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d0dcfe-505a-47f5-8cba-44bfa3446dc6 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:52.066544 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d0dcfe-505a-47f5-8cba-44bfa3446dc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:52.066817 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d0dcfe-505a-47f5-8cba-44bfa3446dc6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:52.077235 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d0dcfe-505a-47f5-8cba-44bfa3446dc6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:24:52.084766 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-54d0dcfe-505a-47f5-8cba-44bfa3446dc6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:24:52.084766 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:24:52.084766 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:24:52.084766 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:24:52.084766 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:24:52.084766 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:24:52.084766 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:24:52.084766 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:24:52.084766 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:24:52.084766 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:24:52.084766 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:24:52.084766 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:24:52.084766 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:24:52.084766 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:24:52.084766 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:24:52.084766 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:24:52.085831 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:24:52.085831 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:24:52.085831 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:24:52.085831 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:24:52.085831 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:24:52.085831 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:24:52.085831 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:24:52.085831 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:24:52.085831 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:24:52.085831 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:24:52.085831 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:24:52.085831 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:24:52.085831 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:24:52.085831 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:24:52.085831 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:24:52.085831 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:24:52.086694 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:24:52.086694 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:24:52.086694 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:24:52.086694 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:24:52.086694 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:24:52.086694 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:24:52.086694 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:24:52.086694 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:24:52.086694 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:24:52.086694 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:24:52.086694 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:24:52.086694 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:24:52.086694 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 582/1180] 10.222.0.99 () {56 vars in 1096 bytes} [Thu Jul 14 15:24:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:24:52.092305 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ad914a-b609-458b-9bd5-f1e56591b796 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:52.189399 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ad914a-b609-458b-9bd5-f1e56591b796 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:52.222900 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a565d30d-1871-4e06-aa5c-eb9e30e76fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:52.223560 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a565d30d-1871-4e06-aa5c-eb9e30e76fc0 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:52.223875 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a565d30d-1871-4e06-aa5c-eb9e30e76fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:52.224152 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a565d30d-1871-4e06-aa5c-eb9e30e76fc0 None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:52.224891 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a565d30d-1871-4e06-aa5c-eb9e30e76fc0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:24:52.225195 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a565d30d-1871-4e06-aa5c-eb9e30e76fc0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:52.226557 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a565d30d-1871-4e06-aa5c-eb9e30e76fc0 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:52.234325 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 599/1181] 10.222.0.99 () {56 vars in 1099 bytes} [Thu Jul 14 15:24:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:24:52.240259 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d0dcfe-505a-47f5-8cba-44bfa3446dc6 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:52.262547 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:24:52.340270 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d0dcfe-505a-47f5-8cba-44bfa3446dc6 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:52.372232 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de22ba46-633e-4d6a-8b65-75a72f4314c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:52.373645 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de22ba46-633e-4d6a-8b65-75a72f4314c7 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:52.374669 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de22ba46-633e-4d6a-8b65-75a72f4314c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:52.375001 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de22ba46-633e-4d6a-8b65-75a72f4314c7 None admin] PATH_INFO: `/v3/users/a0145d725fe444408ad9e526abc93e7d` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:52.389832 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de22ba46-633e-4d6a-8b65-75a72f4314c7 None admin] RBAC: Authorizing `identity:get_user(user_id=a0145d725fe444408ad9e526abc93e7d)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:52.391277 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de22ba46-633e-4d6a-8b65-75a72f4314c7 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:52.393037 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 583/1182] 10.222.0.99 () {56 vars in 1177 bytes} [Thu Jul 14 15:24:52 2022] GET /identity/v3/users/a0145d725fe444408ad9e526abc93e7d => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:24:52.398982 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a565d30d-1871-4e06-aa5c-eb9e30e76fc0 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:52.458885 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:24:52.501750 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a565d30d-1871-4e06-aa5c-eb9e30e76fc0 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:52.534767 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73394b8-ea96-4c4d-97dd-36f2e53bb859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:52.535421 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73394b8-ea96-4c4d-97dd-36f2e53bb859 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:52.535738 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73394b8-ea96-4c4d-97dd-36f2e53bb859 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:52.536023 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73394b8-ea96-4c4d-97dd-36f2e53bb859 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:52.602039 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73394b8-ea96-4c4d-97dd-36f2e53bb859 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:52.603419 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73394b8-ea96-4c4d-97dd-36f2e53bb859 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:52.658382 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:24:52.701227 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 600/1183] 10.222.0.99 () {58 vars in 1282 bytes} [Thu Jul 14 15:24:52 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 15:24:52.707434 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de22ba46-633e-4d6a-8b65-75a72f4314c7 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:52.784110 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:24:52.806056 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de22ba46-633e-4d6a-8b65-75a72f4314c7 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:52.839714 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e62e278-2f45-472d-8f18-53faa26d50a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:52.840581 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e62e278-2f45-472d-8f18-53faa26d50a7 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:52.840869 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e62e278-2f45-472d-8f18-53faa26d50a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:52.841151 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e62e278-2f45-472d-8f18-53faa26d50a7 None admin] PATH_INFO: `/v3/projects/634ba1b149684168afec025912a68525` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:52.842947 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e62e278-2f45-472d-8f18-53faa26d50a7 None admin] RBAC: Authorizing `identity:get_project(project_id=634ba1b149684168afec025912a68525)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:52.844457 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e62e278-2f45-472d-8f18-53faa26d50a7 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:52.846191 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 584/1184] 10.222.0.99 () {56 vars in 1186 bytes} [Thu Jul 14 15:24:52 2022] GET /identity/v3/projects/634ba1b149684168afec025912a68525 => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:24:52.852557 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73394b8-ea96-4c4d-97dd-36f2e53bb859 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:52.948653 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73394b8-ea96-4c4d-97dd-36f2e53bb859 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:52.982301 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89040e9f-1c9b-4c83-b10f-845ccc7bfe61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:52.983621 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89040e9f-1c9b-4c83-b10f-845ccc7bfe61 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:52.983904 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89040e9f-1c9b-4c83-b10f-845ccc7bfe61 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:52.984175 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89040e9f-1c9b-4c83-b10f-845ccc7bfe61 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:52.985008 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89040e9f-1c9b-4c83-b10f-845ccc7bfe61 None admin] RBAC: Adding query filter params (role.id=d0f6ba1314f14d9f9c816bdb5d0c3aac, scope.project.id=634ba1b149684168afec025912a68525, user.id=a0145d725fe444408ad9e526abc93e7d) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:24:52.985320 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89040e9f-1c9b-4c83-b10f-845ccc7bfe61 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:52.986845 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89040e9f-1c9b-4c83-b10f-845ccc7bfe61 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:52.992988 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 601/1185] 10.222.0.99 () {56 vars in 1374 bytes} [Thu Jul 14 15:24:52 2022] GET /identity/v3/role_assignments?user.id=a0145d725fe444408ad9e526abc93e7d&scope.project.id=634ba1b149684168afec025912a68525&role.id=d0f6ba1314f14d9f9c816bdb5d0c3aac => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:24:55.028971 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf785d01-be00-45be-beb4-3be2c1053608 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:55.030057 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf785d01-be00-45be-beb4-3be2c1053608 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:55.030195 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf785d01-be00-45be-beb4-3be2c1053608 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:55.031561 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 585/1186] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:24:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:24:55.036995 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f2c10d-f832-4af0-8f75-fe4f9296ec32 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:55.037387 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f2c10d-f832-4af0-8f75-fe4f9296ec32 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:55.037555 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f2c10d-f832-4af0-8f75-fe4f9296ec32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:55.053820 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-75f2c10d-f832-4af0-8f75-fe4f9296ec32 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:24:55.054993 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-75f2c10d-f832-4af0-8f75-fe4f9296ec32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:24:55.071829 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:24:55.123800 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 602/1187] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:24:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 15:24:55.147445 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf785d01-be00-45be-beb4-3be2c1053608 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:55.245560 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf785d01-be00-45be-beb4-3be2c1053608 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:55.279476 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2ec1f8-f3a8-4a55-b5d2-1bddd1289560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:55.280270 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2ec1f8-f3a8-4a55-b5d2-1bddd1289560 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:55.280559 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2ec1f8-f3a8-4a55-b5d2-1bddd1289560 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:55.280841 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2ec1f8-f3a8-4a55-b5d2-1bddd1289560 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:55.290734 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2ec1f8-f3a8-4a55-b5d2-1bddd1289560 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:24:55.298500 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-5b2ec1f8-f3a8-4a55-b5d2-1bddd1289560 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:24:55.298500 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:24:55.298500 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:24:55.298500 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:24:55.298500 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:24:55.298500 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:24:55.298500 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:24:55.298500 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:24:55.298500 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:24:55.298500 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:24:55.298500 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:24:55.298500 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:24:55.298500 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:24:55.298500 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:24:55.298500 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:24:55.298500 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:24:55.299563 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:24:55.299563 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:24:55.299563 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:24:55.299563 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:24:55.299563 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:24:55.299563 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:24:55.299563 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:24:55.299563 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:24:55.299563 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:24:55.299563 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:24:55.299563 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:24:55.299563 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:24:55.299563 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:24:55.299563 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:24:55.299563 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:24:55.299563 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:24:55.300566 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:24:55.300566 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:24:55.300566 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:24:55.300566 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:24:55.300566 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:24:55.300566 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:24:55.300566 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:24:55.300566 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:24:55.300566 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:24:55.300566 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:24:55.300566 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:24:55.300566 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:24:55.300566 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 586/1188] 10.222.0.99 () {56 vars in 1096 bytes} [Thu Jul 14 15:24:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:24:55.305847 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f2c10d-f832-4af0-8f75-fe4f9296ec32 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:55.412554 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f2c10d-f832-4af0-8f75-fe4f9296ec32 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:55.450571 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9aac322-d995-496d-8c06-7d223a345b6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:55.451883 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aac322-d995-496d-8c06-7d223a345b6d None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:55.452174 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aac322-d995-496d-8c06-7d223a345b6d None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:55.452459 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aac322-d995-496d-8c06-7d223a345b6d None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:55.453217 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9aac322-d995-496d-8c06-7d223a345b6d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:24:55.453527 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9aac322-d995-496d-8c06-7d223a345b6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:55.454912 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9aac322-d995-496d-8c06-7d223a345b6d None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:55.462963 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 603/1189] 10.222.0.99 () {56 vars in 1099 bytes} [Thu Jul 14 15:24:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:24:55.469886 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2ec1f8-f3a8-4a55-b5d2-1bddd1289560 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:55.479906 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:24:55.569668 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2ec1f8-f3a8-4a55-b5d2-1bddd1289560 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:55.602985 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4e68bd-4364-48b6-9fcb-b86b8f9bd5b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:55.603870 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4e68bd-4364-48b6-9fcb-b86b8f9bd5b1 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:55.604155 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4e68bd-4364-48b6-9fcb-b86b8f9bd5b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:55.604431 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4e68bd-4364-48b6-9fcb-b86b8f9bd5b1 None admin] PATH_INFO: `/v3/users/a0145d725fe444408ad9e526abc93e7d` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:55.605615 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4e68bd-4364-48b6-9fcb-b86b8f9bd5b1 None admin] RBAC: Authorizing `identity:get_user(user_id=a0145d725fe444408ad9e526abc93e7d)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:55.607039 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4e68bd-4364-48b6-9fcb-b86b8f9bd5b1 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:55.608795 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 587/1190] 10.222.0.99 () {56 vars in 1177 bytes} [Thu Jul 14 15:24:55 2022] GET /identity/v3/users/a0145d725fe444408ad9e526abc93e7d => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:24:55.614934 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9aac322-d995-496d-8c06-7d223a345b6d None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:55.631360 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:24:55.718360 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9aac322-d995-496d-8c06-7d223a345b6d None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:55.752271 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d20aec-b1f3-4b3f-ae46-b782c9236146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:55.753140 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d20aec-b1f3-4b3f-ae46-b782c9236146 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:55.753425 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d20aec-b1f3-4b3f-ae46-b782c9236146 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:55.753700 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d20aec-b1f3-4b3f-ae46-b782c9236146 None admin] PATH_INFO: `/v3/projects/634ba1b149684168afec025912a68525` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:55.754903 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d20aec-b1f3-4b3f-ae46-b782c9236146 None admin] RBAC: Authorizing `identity:get_project(project_id=634ba1b149684168afec025912a68525)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:55.757221 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d20aec-b1f3-4b3f-ae46-b782c9236146 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:55.759394 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 604/1191] 10.222.0.99 () {56 vars in 1186 bytes} [Thu Jul 14 15:24:55 2022] GET /identity/v3/projects/634ba1b149684168afec025912a68525 => generated 361 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:24:55.766356 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4e68bd-4364-48b6-9fcb-b86b8f9bd5b1 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:55.840305 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:24:55.864413 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4e68bd-4364-48b6-9fcb-b86b8f9bd5b1 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:55.897523 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d0046f-0c31-4fb8-abac-292b98748608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:55.898402 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d0046f-0c31-4fb8-abac-292b98748608 None admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:55.898684 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d0046f-0c31-4fb8-abac-292b98748608 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:55.898960 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d0046f-0c31-4fb8-abac-292b98748608 None admin] PATH_INFO: `/v3/projects/634ba1b149684168afec025912a68525/users/a0145d725fe444408ad9e526abc93e7d/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:55.900902 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d0046f-0c31-4fb8-abac-292b98748608 None admin] RBAC: Authorizing `identity:create_grant(project_id=634ba1b149684168afec025912a68525, user_id=a0145d725fe444408ad9e526abc93e7d, role_id=d0f6ba1314f14d9f9c816bdb5d0c3aac)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:55.902349 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d0046f-0c31-4fb8-abac-292b98748608 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:55.914653 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 588/1192] 10.222.0.99 () {58 vars in 1439 bytes} [Thu Jul 14 15:24:55 2022] PUT /identity/v3/projects/634ba1b149684168afec025912a68525/users/a0145d725fe444408ad9e526abc93e7d/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:24:57.940176 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8f78bb-98b2-45c4-98bb-b4595b604ba7 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:57.940743 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8f78bb-98b2-45c4-98bb-b4595b604ba7 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:57.940743 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8f78bb-98b2-45c4-98bb-b4595b604ba7 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:57.942216 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 605/1193] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:24:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:24:57.947423 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da444f25-9a4c-4579-89a8-4dff3ead4408 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:57.947873 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da444f25-9a4c-4579-89a8-4dff3ead4408 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:57.948035 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da444f25-9a4c-4579-89a8-4dff3ead4408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:57.964452 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-da444f25-9a4c-4579-89a8-4dff3ead4408 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:24:57.965643 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-da444f25-9a4c-4579-89a8-4dff3ead4408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:24:58.034343 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 589/1194] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:24:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 15:24:58.058503 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8f78bb-98b2-45c4-98bb-b4595b604ba7 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:58.142674 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:24:58.162102 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8f78bb-98b2-45c4-98bb-b4595b604ba7 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:58.196552 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3692ad00-4679-404a-93dc-b8b1f14e968e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:58.197265 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3692ad00-4679-404a-93dc-b8b1f14e968e None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:58.197552 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3692ad00-4679-404a-93dc-b8b1f14e968e None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:58.197830 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3692ad00-4679-404a-93dc-b8b1f14e968e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:58.208867 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3692ad00-4679-404a-93dc-b8b1f14e968e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:24:58.217254 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-3692ad00-4679-404a-93dc-b8b1f14e968e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:24:58.217254 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:24:58.217254 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:24:58.217254 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:24:58.217254 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:24:58.217254 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:24:58.217254 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:24:58.217254 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:24:58.217254 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:24:58.217254 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:24:58.217254 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:24:58.217254 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:24:58.217254 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:24:58.217254 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:24:58.217254 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:24:58.217254 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:24:58.218722 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:24:58.218722 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:24:58.218722 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:24:58.218722 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:24:58.218722 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:24:58.218722 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:24:58.218722 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:24:58.218722 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:24:58.218722 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:24:58.218722 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:24:58.218722 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:24:58.218722 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:24:58.218722 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:24:58.218722 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:24:58.218722 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:24:58.218722 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:24:58.220099 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:24:58.220099 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:24:58.220099 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:24:58.220099 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:24:58.220099 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:24:58.220099 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:24:58.220099 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:24:58.220099 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:24:58.220099 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:24:58.220099 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:24:58.220099 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:24:58.220099 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:24:58.220099 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 606/1195] 10.222.0.99 () {56 vars in 1096 bytes} [Thu Jul 14 15:24:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:24:58.227310 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da444f25-9a4c-4579-89a8-4dff3ead4408 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:58.292220 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:24:58.335731 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da444f25-9a4c-4579-89a8-4dff3ead4408 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:58.369294 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9617487e-0ac7-41c9-80c7-45fc527a6203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:58.369954 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9617487e-0ac7-41c9-80c7-45fc527a6203 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:58.370243 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9617487e-0ac7-41c9-80c7-45fc527a6203 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:58.370525 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9617487e-0ac7-41c9-80c7-45fc527a6203 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:58.371271 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9617487e-0ac7-41c9-80c7-45fc527a6203 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:24:58.371582 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9617487e-0ac7-41c9-80c7-45fc527a6203 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:58.373048 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9617487e-0ac7-41c9-80c7-45fc527a6203 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:58.381978 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 590/1196] 10.222.0.99 () {56 vars in 1099 bytes} [Thu Jul 14 15:24:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:24:58.388202 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3692ad00-4679-404a-93dc-b8b1f14e968e None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:58.487050 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3692ad00-4679-404a-93dc-b8b1f14e968e None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:58.520682 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17383c53-9322-4911-8f38-6dd7f61a77f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:58.521550 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17383c53-9322-4911-8f38-6dd7f61a77f5 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:58.521840 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17383c53-9322-4911-8f38-6dd7f61a77f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:58.522122 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17383c53-9322-4911-8f38-6dd7f61a77f5 None admin] PATH_INFO: `/v3/users/a0145d725fe444408ad9e526abc93e7d` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:58.523300 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17383c53-9322-4911-8f38-6dd7f61a77f5 None admin] RBAC: Authorizing `identity:get_user(user_id=a0145d725fe444408ad9e526abc93e7d)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:58.524771 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17383c53-9322-4911-8f38-6dd7f61a77f5 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:58.526370 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 607/1197] 10.222.0.99 () {56 vars in 1177 bytes} [Thu Jul 14 15:24:58 2022] GET /identity/v3/users/a0145d725fe444408ad9e526abc93e7d => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:24:58.533556 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9617487e-0ac7-41c9-80c7-45fc527a6203 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:58.631932 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9617487e-0ac7-41c9-80c7-45fc527a6203 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:58.657675 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:24:58.667448 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6232082d-388a-4aef-aa3a-e9974c52bb3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:58.668133 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6232082d-388a-4aef-aa3a-e9974c52bb3a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:58.668426 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6232082d-388a-4aef-aa3a-e9974c52bb3a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:58.668709 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6232082d-388a-4aef-aa3a-e9974c52bb3a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:58.735382 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6232082d-388a-4aef-aa3a-e9974c52bb3a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:58.736867 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6232082d-388a-4aef-aa3a-e9974c52bb3a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:58.832320 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 591/1198] 10.222.0.99 () {58 vars in 1282 bytes} [Thu Jul 14 15:24:58 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 15:24:58.838472 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17383c53-9322-4911-8f38-6dd7f61a77f5 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:58.850843 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:24:58.937962 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17383c53-9322-4911-8f38-6dd7f61a77f5 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:58.970500 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63943ce1-edaa-47d5-89a7-7a0d5ca83c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:58.971343 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63943ce1-edaa-47d5-89a7-7a0d5ca83c11 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:58.971650 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63943ce1-edaa-47d5-89a7-7a0d5ca83c11 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:58.971927 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63943ce1-edaa-47d5-89a7-7a0d5ca83c11 None admin] PATH_INFO: `/v3/projects/634ba1b149684168afec025912a68525` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:58.973221 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63943ce1-edaa-47d5-89a7-7a0d5ca83c11 None admin] RBAC: Authorizing `identity:get_project(project_id=634ba1b149684168afec025912a68525)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:58.974657 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63943ce1-edaa-47d5-89a7-7a0d5ca83c11 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:58.976350 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 608/1199] 10.222.0.99 () {56 vars in 1186 bytes} [Thu Jul 14 15:24:58 2022] GET /identity/v3/projects/634ba1b149684168afec025912a68525 => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:24:58.982635 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6232082d-388a-4aef-aa3a-e9974c52bb3a None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:24:59.010130 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:24:59.083438 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6232082d-388a-4aef-aa3a-e9974c52bb3a None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:24:59.116336 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee6cdfb-e048-44f3-9c73-52172235cc53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:24:59.116990 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee6cdfb-e048-44f3-9c73-52172235cc53 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:24:59.117278 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee6cdfb-e048-44f3-9c73-52172235cc53 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:24:59.117556 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee6cdfb-e048-44f3-9c73-52172235cc53 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:24:59.118381 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee6cdfb-e048-44f3-9c73-52172235cc53 None admin] RBAC: Adding query filter params (role.id=d0f6ba1314f14d9f9c816bdb5d0c3aac, scope.project.id=634ba1b149684168afec025912a68525, user.id=a0145d725fe444408ad9e526abc93e7d) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:24:59.118695 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee6cdfb-e048-44f3-9c73-52172235cc53 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:24:59.120257 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee6cdfb-e048-44f3-9c73-52172235cc53 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:24:59.126821 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 592/1200] 10.222.0.99 () {56 vars in 1374 bytes} [Thu Jul 14 15:24:58 2022] GET /identity/v3/role_assignments?user.id=a0145d725fe444408ad9e526abc93e7d&scope.project.id=634ba1b149684168afec025912a68525&role.id=d0f6ba1314f14d9f9c816bdb5d0c3aac => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:25:08.966457 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=61 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:25:09.124354 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=55 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:25:56.056633 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4ca5a3-1b79-411d-9cdb-ecd9c5085449 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:25:56.057278 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4ca5a3-1b79-411d-9cdb-ecd9c5085449 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:25:56.057278 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4ca5a3-1b79-411d-9cdb-ecd9c5085449 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:25:56.058791 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 609/1201] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:25:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:25:56.065215 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c75483-435b-40b6-a19c-562bf3a50e43 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:25:56.065747 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c75483-435b-40b6-a19c-562bf3a50e43 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:25:56.065852 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c75483-435b-40b6-a19c-562bf3a50e43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:25:56.093568 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-75c75483-435b-40b6-a19c-562bf3a50e43 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:25:56.095109 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-75c75483-435b-40b6-a19c-562bf3a50e43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:25:56.195640 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 593/1202] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:25:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 15:25:56.216703 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4ca5a3-1b79-411d-9cdb-ecd9c5085449 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:25:56.376062 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4ca5a3-1b79-411d-9cdb-ecd9c5085449 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:25:56.428826 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a2874e-a4a7-4b67-b14b-4ce23d8b13e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:25:56.429824 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a2874e-a4a7-4b67-b14b-4ce23d8b13e9 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:25:56.430154 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a2874e-a4a7-4b67-b14b-4ce23d8b13e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:25:56.430498 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a2874e-a4a7-4b67-b14b-4ce23d8b13e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:25:56.444215 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a2874e-a4a7-4b67-b14b-4ce23d8b13e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:25:56.445892 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a2874e-a4a7-4b67-b14b-4ce23d8b13e9 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:25:56.468745 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-23a2874e-a4a7-4b67-b14b-4ce23d8b13e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:25:56.468745 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:25:56.468745 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:25:56.468745 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:25:56.468745 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:25:56.468745 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:25:56.468745 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:25:56.468745 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:25:56.468745 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:25:56.468745 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:25:56.468745 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:25:56.468745 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:25:56.468745 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:25:56.468745 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:25:56.468745 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:25:56.470326 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:25:56.470326 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:25:56.470326 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:25:56.470326 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:25:56.470326 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:25:56.470326 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:25:56.470326 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:25:56.470326 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:25:56.470326 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:25:56.470326 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:25:56.470326 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:25:56.470326 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:25:56.470326 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:25:56.470326 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:25:56.470326 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:25:56.470326 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:25:56.470326 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:25:56.471848 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:25:56.471848 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:25:56.471848 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:25:56.471848 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:25:56.474000 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 610/1203] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:25:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:25:56.482602 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c75483-435b-40b6-a19c-562bf3a50e43 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:25:56.625872 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c75483-435b-40b6-a19c-562bf3a50e43 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:25:56.663084 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:25:56.671453 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe78911d-1f93-4641-8777-6347285277fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:25:56.672398 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe78911d-1f93-4641-8777-6347285277fa None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:25:56.672866 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe78911d-1f93-4641-8777-6347285277fa None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:25:56.673215 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe78911d-1f93-4641-8777-6347285277fa None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:25:56.674292 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe78911d-1f93-4641-8777-6347285277fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:25:56.674668 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe78911d-1f93-4641-8777-6347285277fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:25:56.676283 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe78911d-1f93-4641-8777-6347285277fa None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:25:56.695647 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 594/1204] 10.222.0.99 () {56 vars in 1109 bytes} [Thu Jul 14 15:25:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:25:56.704614 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a2874e-a4a7-4b67-b14b-4ce23d8b13e9 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:25:56.785707 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:25:56.868960 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a2874e-a4a7-4b67-b14b-4ce23d8b13e9 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:25:56.927513 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e7512f-c230-4117-9d01-dbcb9c1a9e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:25:56.928551 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e7512f-c230-4117-9d01-dbcb9c1a9e1b None admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:25:56.928847 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e7512f-c230-4117-9d01-dbcb9c1a9e1b None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:25:56.929128 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e7512f-c230-4117-9d01-dbcb9c1a9e1b None admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:25:56.934200 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e7512f-c230-4117-9d01-dbcb9c1a9e1b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:25:56.936401 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e7512f-c230-4117-9d01-dbcb9c1a9e1b None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:25:56.975009 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 611/1205] 10.222.0.99 () {60 vars in 1132 bytes} [Thu Jul 14 15:25:56 2022] POST /identity/v3/users => generated 251 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:25:59.283760 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb21d6e-2dc8-4d30-a763-bc13db566ed1 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:25:59.284312 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb21d6e-2dc8-4d30-a763-bc13db566ed1 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:25:59.284312 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb21d6e-2dc8-4d30-a763-bc13db566ed1 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:25:59.285841 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 595/1206] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:25:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:25:59.291785 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05361400-ddb5-4b06-8fc7-01c8fd164ea0 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:25:59.292330 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05361400-ddb5-4b06-8fc7-01c8fd164ea0 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:25:59.292417 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05361400-ddb5-4b06-8fc7-01c8fd164ea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:25:59.318371 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-05361400-ddb5-4b06-8fc7-01c8fd164ea0 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:25:59.319634 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-05361400-ddb5-4b06-8fc7-01c8fd164ea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:25:59.433129 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 612/1207] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:25:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 15:25:59.457176 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb21d6e-2dc8-4d30-a763-bc13db566ed1 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:25:59.602426 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb21d6e-2dc8-4d30-a763-bc13db566ed1 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:25:59.654471 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98aec81-bcfb-44c7-a9ec-7a48808bd0d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:25:59.656692 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98aec81-bcfb-44c7-a9ec-7a48808bd0d8 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:25:59.657224 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98aec81-bcfb-44c7-a9ec-7a48808bd0d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:25:59.657629 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98aec81-bcfb-44c7-a9ec-7a48808bd0d8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:25:59.675109 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98aec81-bcfb-44c7-a9ec-7a48808bd0d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:25:59.686983 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-d98aec81-bcfb-44c7-a9ec-7a48808bd0d8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 15:25:59.686983 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:25:59.686983 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:25:59.686983 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:25:59.686983 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:25:59.686983 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:25:59.686983 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:25:59.686983 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:25:59.686983 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:25:59.686983 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:25:59.686983 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:25:59.686983 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:25:59.686983 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:25:59.686983 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:25:59.686983 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:25:59.686983 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:25:59.688909 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:25:59.688909 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:25:59.688909 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:25:59.688909 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:25:59.688909 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:25:59.688909 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:25:59.688909 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:25:59.688909 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:25:59.688909 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:25:59.688909 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:25:59.688909 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:25:59.688909 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:25:59.688909 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:25:59.688909 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:25:59.688909 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:25:59.688909 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:25:59.690089 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:25:59.690089 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:25:59.690089 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:25:59.690089 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:25:59.690089 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:25:59.690089 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:25:59.690089 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:25:59.690089 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:25:59.690089 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:25:59.690089 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:25:59.690089 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 15:25:59.690089 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:25:59.690089 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 596/1208] 10.222.0.99 () {56 vars in 1102 bytes} [Thu Jul 14 15:25:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:25:59.697197 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05361400-ddb5-4b06-8fc7-01c8fd164ea0 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:25:59.768093 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:25:59.863746 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05361400-ddb5-4b06-8fc7-01c8fd164ea0 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:25:59.923219 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaae344-91d2-4f8b-bd5e-4fca3ace648c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:25:59.924819 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaae344-91d2-4f8b-bd5e-4fca3ace648c None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:25:59.924819 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaae344-91d2-4f8b-bd5e-4fca3ace648c None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:25:59.925697 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaae344-91d2-4f8b-bd5e-4fca3ace648c None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:25:59.927270 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaae344-91d2-4f8b-bd5e-4fca3ace648c None admin] RBAC: Adding query filter params (name=service) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:25:59.928054 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaae344-91d2-4f8b-bd5e-4fca3ace648c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:25:59.930882 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaae344-91d2-4f8b-bd5e-4fca3ace648c None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:25:59.947391 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 613/1209] 10.222.0.99 () {56 vars in 1103 bytes} [Thu Jul 14 15:25:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:25:59.954204 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98aec81-bcfb-44c7-a9ec-7a48808bd0d8 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:26:00.022611 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:26:00.086954 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98aec81-bcfb-44c7-a9ec-7a48808bd0d8 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:26:00.124465 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168b2354-015a-4e47-bd5c-976b5a699b3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:26:00.125181 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168b2354-015a-4e47-bd5c-976b5a699b3a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:26:00.125461 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168b2354-015a-4e47-bd5c-976b5a699b3a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:26:00.125744 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168b2354-015a-4e47-bd5c-976b5a699b3a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:26:00.141965 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168b2354-015a-4e47-bd5c-976b5a699b3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:26:00.143771 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168b2354-015a-4e47-bd5c-976b5a699b3a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:26:00.162870 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-168b2354-015a-4e47-bd5c-976b5a699b3a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:26:00.162870 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:26:00.162870 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:26:00.162870 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:26:00.162870 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:26:00.162870 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:26:00.162870 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:26:00.162870 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:26:00.162870 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:26:00.162870 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:26:00.162870 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:26:00.162870 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:26:00.162870 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:26:00.162870 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:26:00.162870 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:26:00.164102 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:26:00.164102 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:26:00.164102 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:26:00.164102 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:26:00.164102 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:26:00.164102 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:26:00.164102 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:26:00.164102 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:26:00.164102 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:26:00.164102 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:26:00.164102 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:26:00.164102 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:26:00.164102 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:26:00.164102 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:26:00.164102 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:26:00.164102 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:26:00.164102 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:26:00.165173 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:26:00.165173 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:26:00.165173 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:26:00.165173 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:26:00.165173 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 597/1210] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:25:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:26:00.172903 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaae344-91d2-4f8b-bd5e-4fca3ace648c None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:26:00.304234 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:26:00.307589 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaae344-91d2-4f8b-bd5e-4fca3ace648c None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:26:00.351676 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93152ea-4031-4180-a4ff-7b7c4af468ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:26:00.352583 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93152ea-4031-4180-a4ff-7b7c4af468ab None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:26:00.352954 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93152ea-4031-4180-a4ff-7b7c4af468ab None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:26:00.353332 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93152ea-4031-4180-a4ff-7b7c4af468ab None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:26:00.354093 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93152ea-4031-4180-a4ff-7b7c4af468ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:26:00.354401 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93152ea-4031-4180-a4ff-7b7c4af468ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:26:00.355995 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93152ea-4031-4180-a4ff-7b7c4af468ab None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:26:00.371010 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 614/1211] 10.222.0.99 () {56 vars in 1109 bytes} [Thu Jul 14 15:26:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:26:00.377797 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168b2354-015a-4e47-bd5c-976b5a699b3a None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:26:00.531331 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168b2354-015a-4e47-bd5c-976b5a699b3a None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:26:00.533589 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:26:00.577574 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70b7c3c-e457-452e-8f58-cca322973697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:26:00.578557 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70b7c3c-e457-452e-8f58-cca322973697 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:26:00.578867 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70b7c3c-e457-452e-8f58-cca322973697 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:26:00.579147 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70b7c3c-e457-452e-8f58-cca322973697 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:26:00.587814 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70b7c3c-e457-452e-8f58-cca322973697 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:26:00.589638 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70b7c3c-e457-452e-8f58-cca322973697 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:26:00.608879 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-b70b7c3c-e457-452e-8f58-cca322973697 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 14 15:26:00.608879 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:26:00.608879 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:26:00.608879 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:26:00.608879 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:26:00.608879 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:26:00.608879 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:26:00.608879 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:26:00.608879 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:26:00.608879 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:26:00.608879 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:26:00.608879 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:26:00.608879 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 15:26:00.608879 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 15:26:00.608879 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 15:26:00.608879 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 15:26:00.610176 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:26:00.610176 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:26:00.610176 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 15:26:00.610176 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:26:00.610176 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 15:26:00.610176 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:26:00.610176 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:26:00.610176 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:26:00.610176 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:26:00.610176 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:26:00.610176 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:26:00.610176 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:26:00.610176 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:26:00.610176 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:26:00.610176 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:26:00.610176 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:26:00.610176 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:26:00.610176 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:26:00.611354 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:26:00.611354 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:26:00.611354 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 15:26:00.611354 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 15:26:00.611354 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 15:26:00.611354 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 15:26:00.611354 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 15:26:00.611354 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 15:26:00.611354 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 14 15:26:00.611354 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:26:00.611354 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 598/1212] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:26:00 2022] GET /identity/v3/users/placement => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:26:00.619046 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93152ea-4031-4180-a4ff-7b7c4af468ab None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:26:00.772388 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93152ea-4031-4180-a4ff-7b7c4af468ab None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:26:00.817431 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6838d9-0c35-4c1c-844d-42927339584e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:26:00.818348 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6838d9-0c35-4c1c-844d-42927339584e None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:26:00.818745 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6838d9-0c35-4c1c-844d-42927339584e None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:26:00.819102 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6838d9-0c35-4c1c-844d-42927339584e None admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:26:00.820094 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6838d9-0c35-4c1c-844d-42927339584e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:26:00.820493 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6838d9-0c35-4c1c-844d-42927339584e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:26:00.822237 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6838d9-0c35-4c1c-844d-42927339584e None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:26:00.841668 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 615/1213] 10.222.0.99 () {56 vars in 1143 bytes} [Thu Jul 14 15:26:00 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:26:00.848825 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70b7c3c-e457-452e-8f58-cca322973697 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:26:00.982226 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70b7c3c-e457-452e-8f58-cca322973697 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:26:01.036723 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5258df-a452-4b0e-acb3-802e5c1be538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:26:01.037541 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5258df-a452-4b0e-acb3-802e5c1be538 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:26:01.039994 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5258df-a452-4b0e-acb3-802e5c1be538 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:26:01.039994 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5258df-a452-4b0e-acb3-802e5c1be538 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:26:01.091303 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:26:01.146072 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5258df-a452-4b0e-acb3-802e5c1be538 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:26:01.147708 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5258df-a452-4b0e-acb3-802e5c1be538 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:26:01.293806 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 599/1214] 10.222.0.99 () {58 vars in 1282 bytes} [Thu Jul 14 15:26:00 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 15:26:01.300847 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6838d9-0c35-4c1c-844d-42927339584e None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:26:01.323927 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:26:01.429718 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6838d9-0c35-4c1c-844d-42927339584e None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:26:01.477542 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110ca429-79db-4d94-9c4d-bc668376502e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:26:01.478701 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110ca429-79db-4d94-9c4d-bc668376502e None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:26:01.479140 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110ca429-79db-4d94-9c4d-bc668376502e None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:26:01.479711 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110ca429-79db-4d94-9c4d-bc668376502e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:26:01.500222 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110ca429-79db-4d94-9c4d-bc668376502e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:26:01.501726 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110ca429-79db-4d94-9c4d-bc668376502e None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:26:01.523732 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-110ca429-79db-4d94-9c4d-bc668376502e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:26:01.523732 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:26:01.523732 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:26:01.523732 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:26:01.523732 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:26:01.523732 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:26:01.523732 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:26:01.523732 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:26:01.523732 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:26:01.523732 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:26:01.523732 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:26:01.523732 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:26:01.523732 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:26:01.523732 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:26:01.523732 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:26:01.525063 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:26:01.525063 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:26:01.525063 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:26:01.525063 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:26:01.525063 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:26:01.525063 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:26:01.525063 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:26:01.525063 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:26:01.525063 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:26:01.525063 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:26:01.525063 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:26:01.525063 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:26:01.525063 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:26:01.525063 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:26:01.525063 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:26:01.525063 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:26:01.525063 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:26:01.528901 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:26:01.528901 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:26:01.528901 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:26:01.528901 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:26:01.528901 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 616/1215] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:26:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 15:26:01.532009 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5258df-a452-4b0e-acb3-802e5c1be538 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:26:01.636614 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:26:01.721707 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5258df-a452-4b0e-acb3-802e5c1be538 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:26:01.776920 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2572999-be3b-4280-a78b-6696b716b6e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:26:01.777768 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2572999-be3b-4280-a78b-6696b716b6e1 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:26:01.777768 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2572999-be3b-4280-a78b-6696b716b6e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:26:01.778073 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2572999-be3b-4280-a78b-6696b716b6e1 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:26:01.778713 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2572999-be3b-4280-a78b-6696b716b6e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:26:01.779107 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2572999-be3b-4280-a78b-6696b716b6e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:26:01.780568 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2572999-be3b-4280-a78b-6696b716b6e1 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:26:01.803357 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 600/1216] 10.222.0.99 () {56 vars in 1109 bytes} [Thu Jul 14 15:26:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:26:01.809521 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110ca429-79db-4d94-9c4d-bc668376502e None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:26:01.931354 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:26:01.994179 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110ca429-79db-4d94-9c4d-bc668376502e None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:26:02.063761 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b689286c-f0c1-4686-b286-01dc34e92b8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:26:02.064583 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b689286c-f0c1-4686-b286-01dc34e92b8e None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:26:02.064974 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b689286c-f0c1-4686-b286-01dc34e92b8e None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:26:02.065233 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b689286c-f0c1-4686-b286-01dc34e92b8e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:26:02.073325 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b689286c-f0c1-4686-b286-01dc34e92b8e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:26:02.075178 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b689286c-f0c1-4686-b286-01dc34e92b8e None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:26:02.092211 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-b689286c-f0c1-4686-b286-01dc34e92b8e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:26:02.092211 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:26:02.092211 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:26:02.092211 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:26:02.092211 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:26:02.092211 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:26:02.092211 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:26:02.092211 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:26:02.092211 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:26:02.092211 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:26:02.092211 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:26:02.092211 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:26:02.092211 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 15:26:02.092211 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 15:26:02.092211 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 15:26:02.094009 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 15:26:02.094009 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:26:02.094009 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:26:02.094009 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:26:02.094009 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:26:02.094009 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:26:02.094009 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:26:02.094009 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:26:02.094009 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:26:02.094009 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:26:02.094009 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:26:02.094009 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:26:02.094009 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:26:02.094009 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:26:02.094009 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:26:02.094009 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:26:02.094009 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:26:02.095029 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 15:26:02.095029 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 15:26:02.095029 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 15:26:02.095029 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 15:26:02.095029 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 15:26:02.095029 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 15:26:02.095029 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:26:02.095029 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:26:02.100550 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 617/1217] 10.222.0.99 () {56 vars in 1111 bytes} [Thu Jul 14 15:26:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:26:02.101266 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2572999-be3b-4280-a78b-6696b716b6e1 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:26:02.236163 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2572999-be3b-4280-a78b-6696b716b6e1 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:26:02.265484 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:26:02.277455 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d87643-992d-4cfa-8ab8-1000db4187c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:26:02.278127 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d87643-992d-4cfa-8ab8-1000db4187c8 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:26:02.278413 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d87643-992d-4cfa-8ab8-1000db4187c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:26:02.278706 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d87643-992d-4cfa-8ab8-1000db4187c8 None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:26:02.279490 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d87643-992d-4cfa-8ab8-1000db4187c8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:26:02.280346 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d87643-992d-4cfa-8ab8-1000db4187c8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:26:02.281883 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d87643-992d-4cfa-8ab8-1000db4187c8 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:26:02.300943 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 601/1218] 10.222.0.99 () {56 vars in 1148 bytes} [Thu Jul 14 15:26:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:26:02.302373 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b689286c-f0c1-4686-b286-01dc34e92b8e None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:26:02.430426 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b689286c-f0c1-4686-b286-01dc34e92b8e None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:26:02.461905 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:26:02.469311 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78c02cc-9656-4d37-8d7c-1c22a782ec16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:26:02.470064 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78c02cc-9656-4d37-8d7c-1c22a782ec16 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:26:02.470363 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78c02cc-9656-4d37-8d7c-1c22a782ec16 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:26:02.470641 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78c02cc-9656-4d37-8d7c-1c22a782ec16 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:26:02.471639 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78c02cc-9656-4d37-8d7c-1c22a782ec16 None admin] RBAC: Adding query filter params (role.id=74e3b597938f49d095cb424c54b5a811, scope.project.id=9048d1a6f00443ec92f11471babe5a56, user.id=dc157c485f794e968a60d8405c9310ae) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:26:02.471958 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78c02cc-9656-4d37-8d7c-1c22a782ec16 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:26:02.473485 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78c02cc-9656-4d37-8d7c-1c22a782ec16 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:26:02.480110 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 618/1219] 10.222.0.99 () {56 vars in 1374 bytes} [Thu Jul 14 15:26:02 2022] GET /identity/v3/role_assignments?user.id=dc157c485f794e968a60d8405c9310ae&scope.project.id=9048d1a6f00443ec92f11471babe5a56&role.id=74e3b597938f49d095cb424c54b5a811 => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:26:04.628891 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29268f99-3494-470f-8f85-173e0744a3bd None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:26:04.629465 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29268f99-3494-470f-8f85-173e0744a3bd None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:26:04.629465 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29268f99-3494-470f-8f85-173e0744a3bd None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:26:04.630940 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 602/1220] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:26:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 15:26:04.637654 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c438aa0-ec4f-4d93-ad08-567bb473c1a5 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:26:04.638185 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c438aa0-ec4f-4d93-ad08-567bb473c1a5 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:26:04.638287 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c438aa0-ec4f-4d93-ad08-567bb473c1a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:26:04.657306 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-5c438aa0-ec4f-4d93-ad08-567bb473c1a5 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:26:04.658586 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-5c438aa0-ec4f-4d93-ad08-567bb473c1a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:26:04.741848 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 619/1221] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:26:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 15:26:04.770229 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29268f99-3494-470f-8f85-173e0744a3bd None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:26:04.911456 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29268f99-3494-470f-8f85-173e0744a3bd None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:26:04.957517 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9ad058-8b83-4285-9717-c0e772dd5f2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:26:04.958399 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9ad058-8b83-4285-9717-c0e772dd5f2a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:26:04.958686 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9ad058-8b83-4285-9717-c0e772dd5f2a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:26:04.958962 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9ad058-8b83-4285-9717-c0e772dd5f2a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:26:04.971236 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9ad058-8b83-4285-9717-c0e772dd5f2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:26:04.979009 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-aa9ad058-8b83-4285-9717-c0e772dd5f2a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 15:26:04.979009 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:26:04.979009 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:26:04.979009 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:26:04.979009 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:26:04.979009 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:26:04.979009 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:26:04.979009 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:26:04.979009 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:26:04.979009 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:26:04.979009 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:26:04.979009 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:26:04.979009 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:26:04.979009 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:26:04.979009 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:26:04.979009 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:26:04.980566 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:26:04.980566 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:26:04.980566 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:26:04.980566 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:26:04.980566 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:26:04.980566 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:26:04.980566 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:26:04.980566 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:26:04.980566 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:26:04.980566 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:26:04.980566 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:26:04.980566 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:26:04.980566 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:26:04.980566 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:26:04.980566 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:26:04.980566 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:26:04.981995 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:26:04.981995 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:26:04.981995 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:26:04.981995 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:26:04.981995 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:26:04.981995 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:26:04.981995 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:26:04.981995 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:26:04.981995 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:26:04.981995 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:26:04.981995 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 15:26:04.981995 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:26:04.981995 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 603/1222] 10.222.0.99 () {56 vars in 1102 bytes} [Thu Jul 14 15:26:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:26:04.987491 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c438aa0-ec4f-4d93-ad08-567bb473c1a5 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:26:05.117561 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c438aa0-ec4f-4d93-ad08-567bb473c1a5 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:26:05.145309 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:26:05.160787 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92fb9be-5062-4e4a-b213-6f65f4448b5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:26:05.161872 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92fb9be-5062-4e4a-b213-6f65f4448b5d None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:26:05.161998 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92fb9be-5062-4e4a-b213-6f65f4448b5d None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:26:05.162333 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92fb9be-5062-4e4a-b213-6f65f4448b5d None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:26:05.163269 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92fb9be-5062-4e4a-b213-6f65f4448b5d None admin] RBAC: Adding query filter params (name=service) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:26:05.163583 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92fb9be-5062-4e4a-b213-6f65f4448b5d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:26:05.165517 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92fb9be-5062-4e4a-b213-6f65f4448b5d None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:26:05.175645 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 620/1223] 10.222.0.99 () {56 vars in 1103 bytes} [Thu Jul 14 15:26:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:26:05.183101 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9ad058-8b83-4285-9717-c0e772dd5f2a None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:26:05.241578 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:26:05.315406 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9ad058-8b83-4285-9717-c0e772dd5f2a None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:26:05.358961 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bdb9da-2695-4de5-aaef-b664334102ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:26:05.359899 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bdb9da-2695-4de5-aaef-b664334102ba None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:26:05.360198 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bdb9da-2695-4de5-aaef-b664334102ba None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:26:05.360486 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bdb9da-2695-4de5-aaef-b664334102ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:26:05.373933 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bdb9da-2695-4de5-aaef-b664334102ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:26:05.375655 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bdb9da-2695-4de5-aaef-b664334102ba None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:26:05.395865 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-01bdb9da-2695-4de5-aaef-b664334102ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:26:05.395865 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:26:05.395865 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:26:05.395865 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:26:05.395865 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:26:05.395865 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:26:05.395865 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:26:05.395865 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:26:05.395865 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:26:05.395865 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:26:05.395865 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:26:05.395865 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:26:05.395865 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:26:05.395865 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:26:05.395865 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:26:05.396869 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:26:05.396869 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:26:05.396869 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:26:05.396869 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:26:05.396869 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:26:05.396869 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:26:05.396869 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:26:05.396869 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:26:05.396869 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:26:05.396869 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:26:05.396869 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:26:05.396869 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:26:05.396869 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:26:05.396869 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:26:05.396869 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:26:05.396869 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:26:05.396869 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:26:05.397946 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:26:05.397946 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:26:05.397946 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:26:05.397946 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:26:05.397946 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 604/1224] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:26:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:26:05.405267 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92fb9be-5062-4e4a-b213-6f65f4448b5d None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:26:05.524926 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92fb9be-5062-4e4a-b213-6f65f4448b5d None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:26:05.563832 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd83064-e7e1-47d6-aafb-8478e2148104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:26:05.564522 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd83064-e7e1-47d6-aafb-8478e2148104 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:26:05.564823 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd83064-e7e1-47d6-aafb-8478e2148104 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:26:05.565179 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd83064-e7e1-47d6-aafb-8478e2148104 None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:26:05.565911 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd83064-e7e1-47d6-aafb-8478e2148104 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:26:05.566245 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd83064-e7e1-47d6-aafb-8478e2148104 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:26:05.567812 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd83064-e7e1-47d6-aafb-8478e2148104 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:26:05.582120 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 621/1225] 10.222.0.99 () {56 vars in 1109 bytes} [Thu Jul 14 15:26:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:26:05.588988 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bdb9da-2695-4de5-aaef-b664334102ba None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:26:05.692682 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:26:05.700598 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bdb9da-2695-4de5-aaef-b664334102ba None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:26:05.743123 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d52485-9745-4916-a06e-22372fdea3ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:26:05.743978 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d52485-9745-4916-a06e-22372fdea3ac None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:26:05.744357 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d52485-9745-4916-a06e-22372fdea3ac None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:26:05.744556 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d52485-9745-4916-a06e-22372fdea3ac None admin] PATH_INFO: `/v3/users/placement` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:26:05.753227 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d52485-9745-4916-a06e-22372fdea3ac None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:26:05.754726 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d52485-9745-4916-a06e-22372fdea3ac None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:26:05.770239 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-e4d52485-9745-4916-a06e-22372fdea3ac None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 14 15:26:05.770239 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:26:05.770239 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:26:05.770239 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:26:05.770239 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:26:05.770239 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:26:05.770239 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:26:05.770239 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:26:05.770239 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:26:05.770239 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:26:05.770239 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:26:05.770239 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:26:05.770239 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 15:26:05.770239 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 15:26:05.770239 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 15:26:05.770239 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 15:26:05.771370 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:26:05.771370 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:26:05.771370 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 15:26:05.771370 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:26:05.771370 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 15:26:05.771370 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:26:05.771370 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:26:05.771370 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:26:05.771370 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:26:05.771370 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:26:05.771370 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:26:05.771370 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:26:05.771370 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:26:05.771370 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:26:05.771370 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:26:05.771370 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:26:05.771370 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:26:05.771370 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:26:05.773048 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:26:05.773048 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:26:05.773048 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 15:26:05.773048 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 15:26:05.773048 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 15:26:05.773048 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 15:26:05.773048 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 15:26:05.773048 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 15:26:05.773048 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 14 15:26:05.773048 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:26:05.773048 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 605/1226] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:26:05 2022] GET /identity/v3/users/placement => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:26:05.777482 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd83064-e7e1-47d6-aafb-8478e2148104 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:26:05.831693 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:26:05.894995 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd83064-e7e1-47d6-aafb-8478e2148104 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:26:05.934920 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34b7d11-f5e5-4e80-a6d0-03a96e9f89a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:26:05.936185 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34b7d11-f5e5-4e80-a6d0-03a96e9f89a0 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:26:05.936669 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34b7d11-f5e5-4e80-a6d0-03a96e9f89a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:26:05.936898 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34b7d11-f5e5-4e80-a6d0-03a96e9f89a0 None admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:26:05.937973 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34b7d11-f5e5-4e80-a6d0-03a96e9f89a0 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:26:05.938298 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34b7d11-f5e5-4e80-a6d0-03a96e9f89a0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:26:05.939951 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34b7d11-f5e5-4e80-a6d0-03a96e9f89a0 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:26:05.958010 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 622/1227] 10.222.0.99 () {56 vars in 1143 bytes} [Thu Jul 14 15:26:05 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:26:05.964655 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d52485-9745-4916-a06e-22372fdea3ac None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:26:06.088979 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d52485-9745-4916-a06e-22372fdea3ac None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:26:06.130856 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ff4145-32ff-4f41-ba5c-cab980f90e51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:26:06.132024 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ff4145-32ff-4f41-ba5c-cab980f90e51 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:26:06.132322 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ff4145-32ff-4f41-ba5c-cab980f90e51 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:26:06.132644 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ff4145-32ff-4f41-ba5c-cab980f90e51 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:26:06.146979 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ff4145-32ff-4f41-ba5c-cab980f90e51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:26:06.148470 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ff4145-32ff-4f41-ba5c-cab980f90e51 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:26:06.167968 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-d7ff4145-32ff-4f41-ba5c-cab980f90e51 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:26:06.167968 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:26:06.167968 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:26:06.167968 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:26:06.167968 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:26:06.167968 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:26:06.167968 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:26:06.167968 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:26:06.167968 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:26:06.167968 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:26:06.167968 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:26:06.167968 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:26:06.167968 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:26:06.167968 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:26:06.167968 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:26:06.169423 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:26:06.169423 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:26:06.169423 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:26:06.169423 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:26:06.169423 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:26:06.169423 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:26:06.169423 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:26:06.169423 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:26:06.169423 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:26:06.169423 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:26:06.169423 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:26:06.169423 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:26:06.169423 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:26:06.169423 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:26:06.169423 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:26:06.169423 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:26:06.169423 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:26:06.172483 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:26:06.172483 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:26:06.172483 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:26:06.172483 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:26:06.172483 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 606/1228] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:26:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:26:06.177078 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34b7d11-f5e5-4e80-a6d0-03a96e9f89a0 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:26:06.292139 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34b7d11-f5e5-4e80-a6d0-03a96e9f89a0 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:26:06.294560 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:26:06.335843 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd1f5d3-484b-4edf-b33a-8e193e8e24c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:26:06.336605 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd1f5d3-484b-4edf-b33a-8e193e8e24c2 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:26:06.336941 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd1f5d3-484b-4edf-b33a-8e193e8e24c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:26:06.337280 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd1f5d3-484b-4edf-b33a-8e193e8e24c2 None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:26:06.338031 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd1f5d3-484b-4edf-b33a-8e193e8e24c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:26:06.338351 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd1f5d3-484b-4edf-b33a-8e193e8e24c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:26:06.339986 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd1f5d3-484b-4edf-b33a-8e193e8e24c2 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:26:06.356653 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 623/1229] 10.222.0.99 () {56 vars in 1109 bytes} [Thu Jul 14 15:26:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:26:06.363112 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ff4145-32ff-4f41-ba5c-cab980f90e51 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:26:06.373787 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:26:06.476607 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ff4145-32ff-4f41-ba5c-cab980f90e51 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:26:06.512978 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d648426e-f2b3-448a-9f60-aa601c51f27e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:26:06.514438 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d648426e-f2b3-448a-9f60-aa601c51f27e None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:26:06.514755 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d648426e-f2b3-448a-9f60-aa601c51f27e None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:26:06.515195 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d648426e-f2b3-448a-9f60-aa601c51f27e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:26:06.522861 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d648426e-f2b3-448a-9f60-aa601c51f27e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:26:06.524531 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d648426e-f2b3-448a-9f60-aa601c51f27e None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:26:06.540508 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-d648426e-f2b3-448a-9f60-aa601c51f27e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:26:06.540508 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:26:06.540508 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:26:06.540508 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:26:06.540508 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:26:06.540508 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:26:06.540508 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:26:06.540508 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:26:06.540508 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:26:06.540508 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:26:06.540508 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:26:06.540508 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:26:06.540508 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 15:26:06.540508 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 15:26:06.540508 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 15:26:06.541479 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 15:26:06.541479 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:26:06.541479 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:26:06.541479 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:26:06.541479 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:26:06.541479 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:26:06.541479 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:26:06.541479 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:26:06.541479 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:26:06.541479 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:26:06.541479 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:26:06.541479 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:26:06.541479 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:26:06.541479 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:26:06.541479 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:26:06.541479 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:26:06.541479 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:26:06.542889 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 15:26:06.542889 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 15:26:06.542889 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 15:26:06.542889 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 15:26:06.542889 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 15:26:06.542889 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 15:26:06.542889 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:26:06.542889 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:26:06.542889 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 607/1230] 10.222.0.99 () {56 vars in 1111 bytes} [Thu Jul 14 15:26:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:26:06.548303 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd1f5d3-484b-4edf-b33a-8e193e8e24c2 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:26:06.664488 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd1f5d3-484b-4edf-b33a-8e193e8e24c2 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:26:06.705409 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68311ae2-db50-4887-b878-d5ca83835b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:26:06.706221 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68311ae2-db50-4887-b878-d5ca83835b4a None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:26:06.706455 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68311ae2-db50-4887-b878-d5ca83835b4a None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:26:06.706763 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68311ae2-db50-4887-b878-d5ca83835b4a None admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:26:06.707738 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68311ae2-db50-4887-b878-d5ca83835b4a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:26:06.708105 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68311ae2-db50-4887-b878-d5ca83835b4a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:26:06.709664 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68311ae2-db50-4887-b878-d5ca83835b4a None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:26:06.733024 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 624/1231] 10.222.0.99 () {56 vars in 1148 bytes} [Thu Jul 14 15:26:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:26:06.734452 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d648426e-f2b3-448a-9f60-aa601c51f27e None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:26:06.816654 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:26:06.857328 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d648426e-f2b3-448a-9f60-aa601c51f27e None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:26:06.894584 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda5d078-e771-4122-b9a3-a0b7f7207850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:26:06.895478 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda5d078-e771-4122-b9a3-a0b7f7207850 None admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:26:06.895840 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda5d078-e771-4122-b9a3-a0b7f7207850 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:26:06.896072 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda5d078-e771-4122-b9a3-a0b7f7207850 None admin] PATH_INFO: `/v3/projects/9048d1a6f00443ec92f11471babe5a56/users/dc157c485f794e968a60d8405c9310ae/roles/74e3b597938f49d095cb424c54b5a811` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:26:06.911355 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda5d078-e771-4122-b9a3-a0b7f7207850 None admin] RBAC: Authorizing `identity:create_grant(project_id=9048d1a6f00443ec92f11471babe5a56, user_id=dc157c485f794e968a60d8405c9310ae, role_id=74e3b597938f49d095cb424c54b5a811)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:26:06.912876 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda5d078-e771-4122-b9a3-a0b7f7207850 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:26:06.925748 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 608/1232] 10.222.0.99 () {58 vars in 1439 bytes} [Thu Jul 14 15:26:06 2022] PUT /identity/v3/projects/9048d1a6f00443ec92f11471babe5a56/users/dc157c485f794e968a60d8405c9310ae/roles/74e3b597938f49d095cb424c54b5a811 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 15:26:09.033424 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72a7bec-d7f8-4963-b272-0645a646c9f5 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:26:09.033966 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72a7bec-d7f8-4963-b272-0645a646c9f5 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:26:09.033966 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72a7bec-d7f8-4963-b272-0645a646c9f5 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:26:09.035643 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 625/1233] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:26:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:26:09.041791 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cb9aee-a0d2-435f-a3a0-614d4978bfe3 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:26:09.042334 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cb9aee-a0d2-435f-a3a0-614d4978bfe3 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:26:09.042438 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cb9aee-a0d2-435f-a3a0-614d4978bfe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:26:09.059030 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-68cb9aee-a0d2-435f-a3a0-614d4978bfe3 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:26:09.060197 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-68cb9aee-a0d2-435f-a3a0-614d4978bfe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:26:09.132966 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 609/1234] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:26:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 15:26:09.158456 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72a7bec-d7f8-4963-b272-0645a646c9f5 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:26:09.186929 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:26:09.276123 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72a7bec-d7f8-4963-b272-0645a646c9f5 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:26:09.314237 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0179e8d2-0434-4713-8dfb-61b40f71537b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:26:09.315061 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0179e8d2-0434-4713-8dfb-61b40f71537b None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:26:09.315346 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0179e8d2-0434-4713-8dfb-61b40f71537b None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:26:09.315701 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0179e8d2-0434-4713-8dfb-61b40f71537b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:26:09.327297 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0179e8d2-0434-4713-8dfb-61b40f71537b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:26:09.336595 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-0179e8d2-0434-4713-8dfb-61b40f71537b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 15:26:09.336595 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:26:09.336595 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:26:09.336595 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:26:09.336595 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:26:09.336595 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:26:09.336595 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:26:09.336595 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:26:09.336595 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:26:09.336595 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:26:09.336595 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:26:09.336595 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:26:09.336595 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:26:09.336595 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:26:09.336595 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:26:09.336595 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:26:09.337757 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:26:09.337757 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:26:09.337757 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:26:09.337757 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:26:09.337757 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:26:09.337757 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:26:09.337757 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:26:09.337757 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:26:09.337757 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:26:09.337757 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:26:09.337757 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:26:09.337757 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:26:09.337757 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:26:09.337757 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:26:09.337757 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:26:09.337757 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:26:09.344352 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:26:09.344352 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:26:09.344352 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:26:09.344352 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:26:09.344352 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:26:09.344352 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:26:09.344352 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:26:09.344352 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:26:09.344352 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:26:09.344352 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:26:09.344352 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 15:26:09.344352 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:26:09.344352 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 626/1235] 10.222.0.99 () {56 vars in 1102 bytes} [Thu Jul 14 15:26:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:26:09.345039 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cb9aee-a0d2-435f-a3a0-614d4978bfe3 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:26:09.391403 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:26:09.447530 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cb9aee-a0d2-435f-a3a0-614d4978bfe3 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:26:09.488793 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789d184e-55e6-4764-b552-dfd073419ed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:26:09.489511 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789d184e-55e6-4764-b552-dfd073419ed3 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:26:09.489801 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789d184e-55e6-4764-b552-dfd073419ed3 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:26:09.490082 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789d184e-55e6-4764-b552-dfd073419ed3 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:26:09.490855 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789d184e-55e6-4764-b552-dfd073419ed3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:26:09.491170 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789d184e-55e6-4764-b552-dfd073419ed3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:26:09.492599 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789d184e-55e6-4764-b552-dfd073419ed3 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:26:09.502131 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 610/1236] 10.222.0.99 () {56 vars in 1103 bytes} [Thu Jul 14 15:26:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:26:09.507855 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0179e8d2-0434-4713-8dfb-61b40f71537b None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:26:09.604760 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:26:09.622464 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0179e8d2-0434-4713-8dfb-61b40f71537b None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:26:09.659332 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde14584-0e59-4bf3-8ea7-f551513682de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:26:09.660169 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde14584-0e59-4bf3-8ea7-f551513682de None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:26:09.660463 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde14584-0e59-4bf3-8ea7-f551513682de None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:26:09.660743 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde14584-0e59-4bf3-8ea7-f551513682de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:26:09.672904 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde14584-0e59-4bf3-8ea7-f551513682de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:26:09.674548 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde14584-0e59-4bf3-8ea7-f551513682de None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:26:09.694682 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-dde14584-0e59-4bf3-8ea7-f551513682de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:26:09.694682 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:26:09.694682 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:26:09.694682 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:26:09.694682 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:26:09.694682 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:26:09.694682 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:26:09.694682 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:26:09.694682 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:26:09.694682 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:26:09.694682 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:26:09.694682 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:26:09.694682 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:26:09.694682 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:26:09.694682 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:26:09.695677 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:26:09.695677 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:26:09.695677 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:26:09.695677 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:26:09.695677 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:26:09.695677 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:26:09.695677 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:26:09.695677 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:26:09.695677 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:26:09.695677 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:26:09.695677 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:26:09.695677 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:26:09.695677 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:26:09.695677 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:26:09.695677 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:26:09.695677 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:26:09.695677 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:26:09.696593 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:26:09.696593 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:26:09.696593 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:26:09.696593 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:26:09.696593 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 627/1237] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:26:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 15:26:09.702101 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789d184e-55e6-4764-b552-dfd073419ed3 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:26:09.821098 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789d184e-55e6-4764-b552-dfd073419ed3 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:26:09.831172 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:26:09.861496 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781d499b-39b5-4066-b4a2-0c4e7ec38a67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:26:09.862099 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781d499b-39b5-4066-b4a2-0c4e7ec38a67 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:26:09.862389 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781d499b-39b5-4066-b4a2-0c4e7ec38a67 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:26:09.862670 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781d499b-39b5-4066-b4a2-0c4e7ec38a67 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:26:09.863415 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781d499b-39b5-4066-b4a2-0c4e7ec38a67 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:26:09.863842 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781d499b-39b5-4066-b4a2-0c4e7ec38a67 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:26:09.865184 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781d499b-39b5-4066-b4a2-0c4e7ec38a67 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:26:09.878823 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 611/1238] 10.222.0.99 () {56 vars in 1109 bytes} [Thu Jul 14 15:26:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:26:09.885397 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde14584-0e59-4bf3-8ea7-f551513682de None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:26:09.994987 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde14584-0e59-4bf3-8ea7-f551513682de None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:26:10.031400 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c510ef1d-7ceb-44aa-8164-32fdd0de39ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:26:10.032364 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c510ef1d-7ceb-44aa-8164-32fdd0de39ae None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:26:10.032692 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c510ef1d-7ceb-44aa-8164-32fdd0de39ae None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:26:10.032933 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c510ef1d-7ceb-44aa-8164-32fdd0de39ae None admin] PATH_INFO: `/v3/users/placement` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:26:10.037391 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:26:10.042714 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c510ef1d-7ceb-44aa-8164-32fdd0de39ae None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:26:10.045713 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c510ef1d-7ceb-44aa-8164-32fdd0de39ae None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:26:10.062677 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-c510ef1d-7ceb-44aa-8164-32fdd0de39ae None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 14 15:26:10.062677 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:26:10.062677 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:26:10.062677 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:26:10.062677 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:26:10.062677 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:26:10.062677 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:26:10.062677 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:26:10.062677 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:26:10.062677 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:26:10.062677 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:26:10.062677 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:26:10.062677 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 15:26:10.062677 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 15:26:10.062677 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 15:26:10.062677 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 15:26:10.063751 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:26:10.063751 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:26:10.063751 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 15:26:10.063751 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:26:10.063751 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 15:26:10.063751 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:26:10.063751 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:26:10.063751 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:26:10.063751 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:26:10.063751 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:26:10.063751 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:26:10.063751 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:26:10.063751 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:26:10.063751 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:26:10.063751 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:26:10.063751 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:26:10.063751 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:26:10.063751 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:26:10.064906 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:26:10.064906 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:26:10.064906 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 15:26:10.064906 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 15:26:10.064906 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 15:26:10.064906 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 15:26:10.064906 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 15:26:10.064906 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 15:26:10.064906 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 14 15:26:10.064906 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:26:10.064906 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 628/1239] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:26:09 2022] GET /identity/v3/users/placement => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:26:10.070195 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781d499b-39b5-4066-b4a2-0c4e7ec38a67 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:26:10.192977 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781d499b-39b5-4066-b4a2-0c4e7ec38a67 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:26:10.240546 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2517c091-9fd3-45ba-9a1c-af66d4f6ca20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:26:10.241108 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2517c091-9fd3-45ba-9a1c-af66d4f6ca20 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:26:10.241363 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2517c091-9fd3-45ba-9a1c-af66d4f6ca20 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:26:10.241645 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2517c091-9fd3-45ba-9a1c-af66d4f6ca20 None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:26:10.242609 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2517c091-9fd3-45ba-9a1c-af66d4f6ca20 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:26:10.242941 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2517c091-9fd3-45ba-9a1c-af66d4f6ca20 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:26:10.244471 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2517c091-9fd3-45ba-9a1c-af66d4f6ca20 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:26:10.264915 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 612/1240] 10.222.0.99 () {56 vars in 1143 bytes} [Thu Jul 14 15:26:10 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:26:10.270728 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c510ef1d-7ceb-44aa-8164-32fdd0de39ae None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:26:10.386510 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c510ef1d-7ceb-44aa-8164-32fdd0de39ae None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:26:10.426225 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac84ef0-32ef-472a-ab45-65a2d90736f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:26:10.427006 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac84ef0-32ef-472a-ab45-65a2d90736f7 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:26:10.427408 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac84ef0-32ef-472a-ab45-65a2d90736f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:26:10.427780 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac84ef0-32ef-472a-ab45-65a2d90736f7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:26:10.500789 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:26:10.508394 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac84ef0-32ef-472a-ab45-65a2d90736f7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:26:10.509857 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac84ef0-32ef-472a-ab45-65a2d90736f7 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:26:10.644044 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 629/1241] 10.222.0.99 () {58 vars in 1282 bytes} [Thu Jul 14 15:26:10 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 15:26:10.650155 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2517c091-9fd3-45ba-9a1c-af66d4f6ca20 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:26:10.681010 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:26:10.783477 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2517c091-9fd3-45ba-9a1c-af66d4f6ca20 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:26:10.824346 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2734fd7-b584-4c5b-bb12-668516f18eda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:26:10.825181 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2734fd7-b584-4c5b-bb12-668516f18eda None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:26:10.825470 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2734fd7-b584-4c5b-bb12-668516f18eda None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:26:10.825788 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2734fd7-b584-4c5b-bb12-668516f18eda None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:26:10.837938 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2734fd7-b584-4c5b-bb12-668516f18eda None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:26:10.839512 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2734fd7-b584-4c5b-bb12-668516f18eda None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:26:10.859389 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-b2734fd7-b584-4c5b-bb12-668516f18eda None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:26:10.859389 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:26:10.859389 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:26:10.859389 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:26:10.859389 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:26:10.859389 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:26:10.859389 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:26:10.859389 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:26:10.859389 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:26:10.859389 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:26:10.859389 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:26:10.859389 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:26:10.859389 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:26:10.859389 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:26:10.859389 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:26:10.860547 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:26:10.860547 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:26:10.860547 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:26:10.860547 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:26:10.860547 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:26:10.860547 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:26:10.860547 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:26:10.860547 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:26:10.860547 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:26:10.860547 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:26:10.860547 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:26:10.860547 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:26:10.860547 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:26:10.860547 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:26:10.860547 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:26:10.860547 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:26:10.860547 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:26:10.861453 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:26:10.861453 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:26:10.861453 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:26:10.861453 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:26:10.861453 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 613/1242] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:26:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 15:26:10.868048 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac84ef0-32ef-472a-ab45-65a2d90736f7 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:26:10.985355 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:26:11.001150 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac84ef0-32ef-472a-ab45-65a2d90736f7 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:26:11.039194 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3c5b1c-84f2-49ad-bec9-c2046f54c5b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:26:11.039996 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3c5b1c-84f2-49ad-bec9-c2046f54c5b1 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:26:11.040268 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3c5b1c-84f2-49ad-bec9-c2046f54c5b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:26:11.040617 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3c5b1c-84f2-49ad-bec9-c2046f54c5b1 None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:26:11.041405 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3c5b1c-84f2-49ad-bec9-c2046f54c5b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:26:11.041702 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3c5b1c-84f2-49ad-bec9-c2046f54c5b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:26:11.043183 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3c5b1c-84f2-49ad-bec9-c2046f54c5b1 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:26:11.055817 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 630/1243] 10.222.0.99 () {56 vars in 1109 bytes} [Thu Jul 14 15:26:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:26:11.061825 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2734fd7-b584-4c5b-bb12-668516f18eda None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:26:11.159993 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:26:11.188446 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2734fd7-b584-4c5b-bb12-668516f18eda None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:26:11.227552 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bda5907-4729-48a1-9ae3-26dd1391dd91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:26:11.228378 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bda5907-4729-48a1-9ae3-26dd1391dd91 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:26:11.228666 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bda5907-4729-48a1-9ae3-26dd1391dd91 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:26:11.228946 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bda5907-4729-48a1-9ae3-26dd1391dd91 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:26:11.237740 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bda5907-4729-48a1-9ae3-26dd1391dd91 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:26:11.239505 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bda5907-4729-48a1-9ae3-26dd1391dd91 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:26:11.260614 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-9bda5907-4729-48a1-9ae3-26dd1391dd91 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:26:11.260614 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:26:11.260614 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:26:11.260614 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:26:11.260614 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:26:11.260614 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:26:11.260614 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:26:11.260614 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:26:11.260614 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:26:11.260614 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:26:11.260614 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:26:11.260614 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:26:11.260614 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 15:26:11.260614 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 15:26:11.260614 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 15:26:11.261739 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 15:26:11.261739 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:26:11.261739 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:26:11.261739 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:26:11.261739 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:26:11.261739 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:26:11.261739 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:26:11.261739 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:26:11.261739 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:26:11.261739 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:26:11.261739 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:26:11.261739 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:26:11.261739 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:26:11.261739 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:26:11.261739 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:26:11.261739 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:26:11.261739 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:26:11.262907 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 15:26:11.262907 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 15:26:11.262907 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 15:26:11.262907 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 15:26:11.262907 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 15:26:11.262907 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 15:26:11.262907 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:26:11.262907 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:26:11.262907 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 614/1244] 10.222.0.99 () {56 vars in 1111 bytes} [Thu Jul 14 15:26:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 15:26:11.269797 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3c5b1c-84f2-49ad-bec9-c2046f54c5b1 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:26:11.399880 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3c5b1c-84f2-49ad-bec9-c2046f54c5b1 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:26:11.445205 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3747893f-c90d-4e74-9842-28d2acf30452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:26:11.446414 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3747893f-c90d-4e74-9842-28d2acf30452 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:26:11.447056 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3747893f-c90d-4e74-9842-28d2acf30452 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:26:11.447646 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3747893f-c90d-4e74-9842-28d2acf30452 None admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:26:11.449644 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3747893f-c90d-4e74-9842-28d2acf30452 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:26:11.450531 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3747893f-c90d-4e74-9842-28d2acf30452 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:26:11.453215 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3747893f-c90d-4e74-9842-28d2acf30452 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:26:11.468317 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 631/1245] 10.222.0.99 () {56 vars in 1148 bytes} [Thu Jul 14 15:26:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:26:11.474856 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bda5907-4729-48a1-9ae3-26dd1391dd91 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:26:11.590616 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bda5907-4729-48a1-9ae3-26dd1391dd91 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:26:11.630505 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dd24f8-3c4b-4877-82a9-67fedbac697e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:26:11.631207 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dd24f8-3c4b-4877-82a9-67fedbac697e None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:26:11.631495 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dd24f8-3c4b-4877-82a9-67fedbac697e None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:26:11.631798 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dd24f8-3c4b-4877-82a9-67fedbac697e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:26:11.632637 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dd24f8-3c4b-4877-82a9-67fedbac697e None admin] RBAC: Adding query filter params (role.id=74e3b597938f49d095cb424c54b5a811, scope.project.id=9048d1a6f00443ec92f11471babe5a56, user.id=dc157c485f794e968a60d8405c9310ae) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:26:11.632952 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dd24f8-3c4b-4877-82a9-67fedbac697e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:26:11.636055 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dd24f8-3c4b-4877-82a9-67fedbac697e None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:26:11.641431 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 615/1246] 10.222.0.99 () {56 vars in 1374 bytes} [Thu Jul 14 15:26:11 2022] GET /identity/v3/role_assignments?user.id=dc157c485f794e968a60d8405c9310ae&scope.project.id=9048d1a6f00443ec92f11471babe5a56&role.id=74e3b597938f49d095cb424c54b5a811 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:26:14.071955 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e26d1c-ed5a-4c7a-a5e5-5c939e62eb6c None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:26:14.072561 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e26d1c-ed5a-4c7a-a5e5-5c939e62eb6c None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:26:14.072771 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e26d1c-ed5a-4c7a-a5e5-5c939e62eb6c None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:26:14.075148 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 632/1247] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:26:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:26:14.082201 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d25c4a4-eecc-45d9-9587-256e9a656f8b None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:26:14.082745 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d25c4a4-eecc-45d9-9587-256e9a656f8b None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:26:14.082745 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d25c4a4-eecc-45d9-9587-256e9a656f8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:26:14.088127 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:26:14.103799 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-0d25c4a4-eecc-45d9-9587-256e9a656f8b None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:26:14.105764 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0d25c4a4-eecc-45d9-9587-256e9a656f8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:26:14.199640 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 616/1248] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:26:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 15:26:14.226916 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e26d1c-ed5a-4c7a-a5e5-5c939e62eb6c None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:26:14.250027 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:26:14.364110 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e26d1c-ed5a-4c7a-a5e5-5c939e62eb6c None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:26:14.409880 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f59ede3-77b4-40d8-8cb3-1ae76b9e711b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:26:14.411976 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f59ede3-77b4-40d8-8cb3-1ae76b9e711b None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:26:14.411976 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f59ede3-77b4-40d8-8cb3-1ae76b9e711b None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:26:14.411976 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f59ede3-77b4-40d8-8cb3-1ae76b9e711b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:26:14.434865 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f59ede3-77b4-40d8-8cb3-1ae76b9e711b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:26:14.443753 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-8f59ede3-77b4-40d8-8cb3-1ae76b9e711b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:26:14.443753 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:26:14.443753 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:26:14.443753 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:26:14.443753 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:26:14.443753 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:26:14.443753 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:26:14.443753 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:26:14.443753 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:26:14.443753 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:26:14.443753 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:26:14.443753 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:26:14.443753 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:26:14.443753 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:26:14.443753 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:26:14.443753 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:26:14.445164 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:26:14.445164 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:26:14.445164 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:26:14.445164 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:26:14.445164 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:26:14.445164 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:26:14.445164 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:26:14.445164 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:26:14.445164 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:26:14.445164 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:26:14.445164 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:26:14.445164 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:26:14.445164 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:26:14.445164 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:26:14.445164 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:26:14.445164 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:26:14.446110 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:26:14.446110 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:26:14.446110 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:26:14.446110 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:26:14.446110 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:26:14.446110 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:26:14.446110 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:26:14.446110 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:26:14.446110 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:26:14.446110 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:26:14.446110 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:26:14.446110 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:26:14.446110 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 633/1249] 10.222.0.99 () {56 vars in 1096 bytes} [Thu Jul 14 15:26:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:26:14.454256 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d25c4a4-eecc-45d9-9587-256e9a656f8b None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:26:14.587333 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d25c4a4-eecc-45d9-9587-256e9a656f8b None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:26:14.637412 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:26:14.644942 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f35e847-0953-4bfb-aa16-e77ccdb0736d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:26:14.645850 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f35e847-0953-4bfb-aa16-e77ccdb0736d None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:26:14.646144 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f35e847-0953-4bfb-aa16-e77ccdb0736d None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:26:14.646416 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f35e847-0953-4bfb-aa16-e77ccdb0736d None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:26:14.647342 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f35e847-0953-4bfb-aa16-e77ccdb0736d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:26:14.647637 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f35e847-0953-4bfb-aa16-e77ccdb0736d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:26:14.649284 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f35e847-0953-4bfb-aa16-e77ccdb0736d None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:26:14.660157 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 617/1250] 10.222.0.99 () {56 vars in 1099 bytes} [Thu Jul 14 15:26:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:26:14.666731 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f59ede3-77b4-40d8-8cb3-1ae76b9e711b None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:26:14.766374 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:26:14.798688 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f59ede3-77b4-40d8-8cb3-1ae76b9e711b None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:26:14.843923 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d8cafa-8239-49fa-8b48-60ec23a7c143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:26:14.844826 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d8cafa-8239-49fa-8b48-60ec23a7c143 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:26:14.845115 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d8cafa-8239-49fa-8b48-60ec23a7c143 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:26:14.845426 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d8cafa-8239-49fa-8b48-60ec23a7c143 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:26:14.860551 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d8cafa-8239-49fa-8b48-60ec23a7c143 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:26:14.862197 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d8cafa-8239-49fa-8b48-60ec23a7c143 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:26:14.882013 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-72d8cafa-8239-49fa-8b48-60ec23a7c143 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:26:14.882013 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:26:14.882013 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:26:14.882013 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:26:14.882013 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:26:14.882013 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:26:14.882013 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:26:14.882013 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:26:14.882013 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:26:14.882013 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:26:14.882013 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:26:14.882013 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:26:14.882013 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:26:14.882013 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:26:14.882013 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:26:14.883200 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:26:14.883200 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:26:14.883200 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:26:14.883200 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:26:14.883200 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:26:14.883200 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:26:14.883200 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:26:14.883200 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:26:14.883200 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:26:14.883200 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:26:14.883200 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:26:14.883200 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:26:14.883200 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:26:14.883200 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:26:14.883200 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:26:14.883200 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:26:14.883200 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:26:14.884478 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:26:14.884478 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:26:14.884478 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:26:14.884478 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:26:14.884478 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 634/1251] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:26:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 15:26:14.891311 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f35e847-0953-4bfb-aa16-e77ccdb0736d None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:26:15.020991 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f35e847-0953-4bfb-aa16-e77ccdb0736d None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:26:15.068805 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77644a97-d7bb-4e28-9051-d2182867294a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:26:15.069615 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77644a97-d7bb-4e28-9051-d2182867294a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:26:15.069897 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77644a97-d7bb-4e28-9051-d2182867294a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:26:15.070178 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77644a97-d7bb-4e28-9051-d2182867294a None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:26:15.071240 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77644a97-d7bb-4e28-9051-d2182867294a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:26:15.071554 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77644a97-d7bb-4e28-9051-d2182867294a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:26:15.072960 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77644a97-d7bb-4e28-9051-d2182867294a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:26:15.094081 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 618/1252] 10.222.0.99 () {56 vars in 1109 bytes} [Thu Jul 14 15:26:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:26:15.100520 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d8cafa-8239-49fa-8b48-60ec23a7c143 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:26:15.213412 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d8cafa-8239-49fa-8b48-60ec23a7c143 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:26:15.245054 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:26:15.257044 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685b5e80-9263-4efe-ac94-5a125e3ea87c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:26:15.258016 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685b5e80-9263-4efe-ac94-5a125e3ea87c None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:26:15.258318 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685b5e80-9263-4efe-ac94-5a125e3ea87c None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:26:15.258600 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685b5e80-9263-4efe-ac94-5a125e3ea87c None admin] PATH_INFO: `/v3/users/placement` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:26:15.269271 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685b5e80-9263-4efe-ac94-5a125e3ea87c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:26:15.270758 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685b5e80-9263-4efe-ac94-5a125e3ea87c None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:26:15.290504 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-685b5e80-9263-4efe-ac94-5a125e3ea87c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 14 15:26:15.290504 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:26:15.290504 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:26:15.290504 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:26:15.290504 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:26:15.290504 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:26:15.290504 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:26:15.290504 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:26:15.290504 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:26:15.290504 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:26:15.290504 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:26:15.290504 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:26:15.290504 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 15:26:15.290504 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 15:26:15.290504 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 15:26:15.290504 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 15:26:15.291765 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:26:15.291765 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:26:15.291765 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 15:26:15.291765 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:26:15.291765 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 15:26:15.291765 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:26:15.291765 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:26:15.291765 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:26:15.291765 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:26:15.291765 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:26:15.291765 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:26:15.291765 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:26:15.291765 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:26:15.291765 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:26:15.291765 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:26:15.291765 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:26:15.291765 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:26:15.291765 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:26:15.293006 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:26:15.293006 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:26:15.293006 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 15:26:15.293006 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 15:26:15.293006 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 15:26:15.293006 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 15:26:15.293006 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 15:26:15.293006 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 15:26:15.293006 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 14 15:26:15.293006 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:26:15.293006 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 635/1253] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:26:15 2022] GET /identity/v3/users/placement => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:26:15.298192 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77644a97-d7bb-4e28-9051-d2182867294a None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:26:15.363347 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:26:15.420822 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77644a97-d7bb-4e28-9051-d2182867294a None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:26:15.464237 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b848981c-1bb2-4229-8203-f413fac31d6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:26:15.464921 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b848981c-1bb2-4229-8203-f413fac31d6d None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:26:15.465212 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b848981c-1bb2-4229-8203-f413fac31d6d None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:26:15.465627 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b848981c-1bb2-4229-8203-f413fac31d6d None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:26:15.466606 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b848981c-1bb2-4229-8203-f413fac31d6d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:26:15.467652 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b848981c-1bb2-4229-8203-f413fac31d6d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:26:15.469221 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b848981c-1bb2-4229-8203-f413fac31d6d None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:26:15.490291 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 619/1254] 10.222.0.99 () {56 vars in 1143 bytes} [Thu Jul 14 15:26:15 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:26:15.497099 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685b5e80-9263-4efe-ac94-5a125e3ea87c None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:26:15.619505 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685b5e80-9263-4efe-ac94-5a125e3ea87c None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:26:15.661892 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c87b19-ba3c-4698-8d11-d5584fad3120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:26:15.662639 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c87b19-ba3c-4698-8d11-d5584fad3120 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:26:15.662939 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c87b19-ba3c-4698-8d11-d5584fad3120 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:26:15.663220 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c87b19-ba3c-4698-8d11-d5584fad3120 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:26:15.734401 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:26:15.756055 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c87b19-ba3c-4698-8d11-d5584fad3120 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:26:15.757976 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c87b19-ba3c-4698-8d11-d5584fad3120 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:26:15.885812 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 636/1255] 10.222.0.99 () {58 vars in 1282 bytes} [Thu Jul 14 15:26:15 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 15:26:15.893841 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b848981c-1bb2-4229-8203-f413fac31d6d None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:26:16.018308 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b848981c-1bb2-4229-8203-f413fac31d6d None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:26:16.034697 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:26:16.063964 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a254af-9bf0-46f6-82fb-1618777311da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:26:16.063964 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a254af-9bf0-46f6-82fb-1618777311da None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:26:16.065181 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a254af-9bf0-46f6-82fb-1618777311da None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:26:16.065590 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a254af-9bf0-46f6-82fb-1618777311da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:26:16.080209 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a254af-9bf0-46f6-82fb-1618777311da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:26:16.081667 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a254af-9bf0-46f6-82fb-1618777311da None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:26:16.100452 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-99a254af-9bf0-46f6-82fb-1618777311da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:26:16.100452 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:26:16.100452 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:26:16.100452 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:26:16.100452 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:26:16.100452 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:26:16.100452 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:26:16.100452 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:26:16.100452 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:26:16.100452 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:26:16.100452 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:26:16.100452 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:26:16.100452 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:26:16.100452 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:26:16.100452 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:26:16.101911 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:26:16.101911 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:26:16.101911 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:26:16.101911 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:26:16.101911 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:26:16.101911 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:26:16.101911 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:26:16.101911 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:26:16.101911 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:26:16.101911 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:26:16.101911 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:26:16.101911 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:26:16.101911 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:26:16.101911 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:26:16.101911 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:26:16.101911 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:26:16.101911 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:26:16.103006 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:26:16.103006 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:26:16.103006 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:26:16.103006 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:26:16.103006 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 620/1256] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:26:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:26:16.107946 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c87b19-ba3c-4698-8d11-d5584fad3120 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:26:16.207177 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:26:16.220563 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c87b19-ba3c-4698-8d11-d5584fad3120 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:26:16.261603 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d4701e-0322-49ca-a856-4df96218bd71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:26:16.262282 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d4701e-0322-49ca-a856-4df96218bd71 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:26:16.262800 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d4701e-0322-49ca-a856-4df96218bd71 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:26:16.263146 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d4701e-0322-49ca-a856-4df96218bd71 None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:26:16.264048 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d4701e-0322-49ca-a856-4df96218bd71 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:26:16.264566 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d4701e-0322-49ca-a856-4df96218bd71 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:26:16.266014 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d4701e-0322-49ca-a856-4df96218bd71 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:26:16.280072 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 637/1257] 10.222.0.99 () {56 vars in 1109 bytes} [Thu Jul 14 15:26:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:26:16.285781 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a254af-9bf0-46f6-82fb-1618777311da None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:26:16.419614 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a254af-9bf0-46f6-82fb-1618777311da None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:26:16.467579 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04844527-ca9d-4641-8e08-f95efc517a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:26:16.468765 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04844527-ca9d-4641-8e08-f95efc517a8b None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:26:16.469119 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04844527-ca9d-4641-8e08-f95efc517a8b None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:26:16.469446 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04844527-ca9d-4641-8e08-f95efc517a8b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:26:16.480362 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04844527-ca9d-4641-8e08-f95efc517a8b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:26:16.482051 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04844527-ca9d-4641-8e08-f95efc517a8b None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:26:16.501074 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-04844527-ca9d-4641-8e08-f95efc517a8b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:26:16.501074 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:26:16.501074 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:26:16.501074 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:26:16.501074 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:26:16.501074 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:26:16.501074 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:26:16.501074 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:26:16.501074 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:26:16.501074 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:26:16.501074 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:26:16.501074 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:26:16.501074 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 15:26:16.501074 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 15:26:16.501074 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 15:26:16.502193 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 15:26:16.502193 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:26:16.502193 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:26:16.502193 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:26:16.502193 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:26:16.502193 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:26:16.502193 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:26:16.502193 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:26:16.502193 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:26:16.502193 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:26:16.502193 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:26:16.502193 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:26:16.502193 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:26:16.502193 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:26:16.502193 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:26:16.502193 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:26:16.502193 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:26:16.508235 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 15:26:16.508235 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 15:26:16.508235 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 15:26:16.508235 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 15:26:16.508235 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 15:26:16.508235 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 15:26:16.508235 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:26:16.508235 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:26:16.508235 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 621/1258] 10.222.0.99 () {56 vars in 1111 bytes} [Thu Jul 14 15:26:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:26:16.511470 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d4701e-0322-49ca-a856-4df96218bd71 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:26:16.626187 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d4701e-0322-49ca-a856-4df96218bd71 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:26:16.666960 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f718739-475e-40ab-9ad5-8d3345bb8ce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:26:16.667716 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f718739-475e-40ab-9ad5-8d3345bb8ce2 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:26:16.668008 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f718739-475e-40ab-9ad5-8d3345bb8ce2 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:26:16.668281 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f718739-475e-40ab-9ad5-8d3345bb8ce2 None admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:26:16.669116 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f718739-475e-40ab-9ad5-8d3345bb8ce2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:26:16.669422 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f718739-475e-40ab-9ad5-8d3345bb8ce2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:26:16.671808 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f718739-475e-40ab-9ad5-8d3345bb8ce2 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:26:16.685197 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 638/1259] 10.222.0.99 () {56 vars in 1148 bytes} [Thu Jul 14 15:26:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:26:16.691971 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04844527-ca9d-4641-8e08-f95efc517a8b None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:26:16.723712 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:26:16.834658 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04844527-ca9d-4641-8e08-f95efc517a8b None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:26:16.874638 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90960361-93e3-4637-9595-b49b14c47a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:26:16.875348 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90960361-93e3-4637-9595-b49b14c47a78 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:26:16.875665 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90960361-93e3-4637-9595-b49b14c47a78 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:26:16.875948 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90960361-93e3-4637-9595-b49b14c47a78 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:26:16.876916 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90960361-93e3-4637-9595-b49b14c47a78 None admin] RBAC: Adding query filter params (role.id=d0f6ba1314f14d9f9c816bdb5d0c3aac, scope.project.id=9048d1a6f00443ec92f11471babe5a56, user.id=dc157c485f794e968a60d8405c9310ae) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:26:16.877258 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90960361-93e3-4637-9595-b49b14c47a78 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:26:16.878796 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90960361-93e3-4637-9595-b49b14c47a78 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:26:16.884941 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 622/1260] 10.222.0.99 () {56 vars in 1374 bytes} [Thu Jul 14 15:26:16 2022] GET /identity/v3/role_assignments?user.id=dc157c485f794e968a60d8405c9310ae&scope.project.id=9048d1a6f00443ec92f11471babe5a56&role.id=d0f6ba1314f14d9f9c816bdb5d0c3aac => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:26:18.975139 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5c1f2a-d8d4-481b-bb4d-b5a036262680 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:26:18.975757 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5c1f2a-d8d4-481b-bb4d-b5a036262680 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:26:18.975757 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5c1f2a-d8d4-481b-bb4d-b5a036262680 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:26:18.977084 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 639/1261] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:26:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:26:18.982289 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328f5edc-7a9c-45d4-bb0e-bc1604ac065e None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:26:18.983263 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328f5edc-7a9c-45d4-bb0e-bc1604ac065e None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:26:18.983576 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328f5edc-7a9c-45d4-bb0e-bc1604ac065e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:26:19.002563 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-328f5edc-7a9c-45d4-bb0e-bc1604ac065e None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:26:19.003799 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-328f5edc-7a9c-45d4-bb0e-bc1604ac065e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:26:19.069978 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 623/1262] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:26:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 15:26:19.093263 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5c1f2a-d8d4-481b-bb4d-b5a036262680 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:26:19.104214 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:26:19.195784 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5c1f2a-d8d4-481b-bb4d-b5a036262680 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:26:19.228346 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3adf4d7-9589-4a80-bbe8-63f3bf040c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:26:19.229101 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3adf4d7-9589-4a80-bbe8-63f3bf040c48 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:26:19.229385 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3adf4d7-9589-4a80-bbe8-63f3bf040c48 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:26:19.229656 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3adf4d7-9589-4a80-bbe8-63f3bf040c48 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:26:19.240232 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3adf4d7-9589-4a80-bbe8-63f3bf040c48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:26:19.247959 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-e3adf4d7-9589-4a80-bbe8-63f3bf040c48 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:26:19.247959 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:26:19.247959 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:26:19.247959 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:26:19.247959 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:26:19.247959 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:26:19.247959 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:26:19.247959 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:26:19.247959 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:26:19.247959 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:26:19.247959 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:26:19.247959 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:26:19.247959 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:26:19.247959 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:26:19.247959 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:26:19.247959 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:26:19.249518 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:26:19.249518 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:26:19.249518 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:26:19.249518 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:26:19.249518 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:26:19.249518 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:26:19.249518 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:26:19.249518 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:26:19.249518 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:26:19.249518 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:26:19.249518 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:26:19.249518 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:26:19.249518 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:26:19.249518 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:26:19.249518 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:26:19.249518 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:26:19.250559 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:26:19.250559 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:26:19.250559 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:26:19.250559 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:26:19.250559 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:26:19.250559 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:26:19.250559 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:26:19.250559 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:26:19.250559 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:26:19.250559 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:26:19.250559 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:26:19.250559 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:26:19.250559 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 640/1263] 10.222.0.99 () {56 vars in 1096 bytes} [Thu Jul 14 15:26:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 15:26:19.257872 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328f5edc-7a9c-45d4-bb0e-bc1604ac065e None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:26:19.275914 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:26:19.361097 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328f5edc-7a9c-45d4-bb0e-bc1604ac065e None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:26:19.395153 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c15a41-28ce-4d85-9304-de730da02019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:26:19.395856 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c15a41-28ce-4d85-9304-de730da02019 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:26:19.396148 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c15a41-28ce-4d85-9304-de730da02019 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:26:19.396432 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c15a41-28ce-4d85-9304-de730da02019 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:26:19.397179 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c15a41-28ce-4d85-9304-de730da02019 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:26:19.397491 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c15a41-28ce-4d85-9304-de730da02019 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:26:19.398867 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c15a41-28ce-4d85-9304-de730da02019 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:26:19.407146 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 624/1264] 10.222.0.99 () {56 vars in 1099 bytes} [Thu Jul 14 15:26:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:26:19.413041 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3adf4d7-9589-4a80-bbe8-63f3bf040c48 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:26:19.479867 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:26:19.514159 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3adf4d7-9589-4a80-bbe8-63f3bf040c48 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:26:19.547537 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f91f65e-5470-45fe-a00d-8d6c356a94e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:26:19.548353 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f91f65e-5470-45fe-a00d-8d6c356a94e2 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:26:19.548642 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f91f65e-5470-45fe-a00d-8d6c356a94e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:26:19.548923 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f91f65e-5470-45fe-a00d-8d6c356a94e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:26:19.559268 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f91f65e-5470-45fe-a00d-8d6c356a94e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:26:19.560798 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f91f65e-5470-45fe-a00d-8d6c356a94e2 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:26:19.578934 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-8f91f65e-5470-45fe-a00d-8d6c356a94e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:26:19.578934 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:26:19.578934 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:26:19.578934 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:26:19.578934 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:26:19.578934 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:26:19.578934 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:26:19.578934 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:26:19.578934 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:26:19.578934 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:26:19.578934 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:26:19.578934 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:26:19.578934 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:26:19.578934 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:26:19.578934 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:26:19.580026 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:26:19.580026 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:26:19.580026 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:26:19.580026 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:26:19.580026 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:26:19.580026 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:26:19.580026 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:26:19.580026 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:26:19.580026 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:26:19.580026 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:26:19.580026 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:26:19.580026 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:26:19.580026 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:26:19.580026 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:26:19.580026 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:26:19.580026 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:26:19.580026 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:26:19.581049 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:26:19.581049 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:26:19.581049 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:26:19.581049 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:26:19.581049 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 641/1265] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:26:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 15:26:19.586125 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c15a41-28ce-4d85-9304-de730da02019 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:26:19.666655 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:26:19.686422 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c15a41-28ce-4d85-9304-de730da02019 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:26:19.724625 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fe2720-f7a2-45ce-ac7c-49755ef32f2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:26:19.725410 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fe2720-f7a2-45ce-ac7c-49755ef32f2c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:26:19.725805 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fe2720-f7a2-45ce-ac7c-49755ef32f2c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:26:19.726115 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fe2720-f7a2-45ce-ac7c-49755ef32f2c None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:26:19.727002 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fe2720-f7a2-45ce-ac7c-49755ef32f2c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:26:19.727401 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fe2720-f7a2-45ce-ac7c-49755ef32f2c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:26:19.729118 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fe2720-f7a2-45ce-ac7c-49755ef32f2c None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:26:19.744381 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 625/1266] 10.222.0.99 () {56 vars in 1109 bytes} [Thu Jul 14 15:26:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:26:19.751490 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f91f65e-5470-45fe-a00d-8d6c356a94e2 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:26:19.854758 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f91f65e-5470-45fe-a00d-8d6c356a94e2 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:26:19.874861 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:26:19.889676 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928c2f3a-6481-4738-902b-45a9c207eb5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:26:19.890483 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928c2f3a-6481-4738-902b-45a9c207eb5d None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:26:19.890771 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928c2f3a-6481-4738-902b-45a9c207eb5d None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:26:19.891051 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928c2f3a-6481-4738-902b-45a9c207eb5d None admin] PATH_INFO: `/v3/users/placement` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:26:19.898544 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928c2f3a-6481-4738-902b-45a9c207eb5d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:26:19.900011 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928c2f3a-6481-4738-902b-45a9c207eb5d None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:26:19.914770 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-928c2f3a-6481-4738-902b-45a9c207eb5d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 14 15:26:19.914770 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:26:19.914770 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:26:19.914770 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:26:19.914770 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:26:19.914770 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:26:19.914770 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:26:19.914770 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:26:19.914770 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:26:19.914770 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:26:19.914770 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:26:19.914770 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:26:19.914770 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 15:26:19.914770 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 15:26:19.914770 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 15:26:19.914770 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 15:26:19.916147 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:26:19.916147 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:26:19.916147 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 15:26:19.916147 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:26:19.916147 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 15:26:19.916147 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:26:19.916147 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:26:19.916147 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:26:19.916147 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:26:19.916147 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:26:19.916147 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:26:19.916147 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:26:19.916147 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:26:19.916147 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:26:19.916147 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:26:19.916147 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:26:19.916147 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:26:19.916147 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:26:19.917446 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:26:19.917446 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:26:19.917446 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 15:26:19.917446 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 15:26:19.917446 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 15:26:19.917446 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 15:26:19.917446 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 15:26:19.917446 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 15:26:19.917446 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 14 15:26:19.917446 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:26:19.917446 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 642/1267] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:26:19 2022] GET /identity/v3/users/placement => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:26:19.922168 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fe2720-f7a2-45ce-ac7c-49755ef32f2c None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:26:20.023944 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fe2720-f7a2-45ce-ac7c-49755ef32f2c None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:26:20.057493 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aae985f-c78b-4769-b90f-88104414ccbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:26:20.058160 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aae985f-c78b-4769-b90f-88104414ccbd None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:26:20.058447 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aae985f-c78b-4769-b90f-88104414ccbd None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:26:20.058725 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aae985f-c78b-4769-b90f-88104414ccbd None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:26:20.059553 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aae985f-c78b-4769-b90f-88104414ccbd None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:26:20.059892 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aae985f-c78b-4769-b90f-88104414ccbd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:26:20.061454 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aae985f-c78b-4769-b90f-88104414ccbd None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:26:20.069823 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:26:20.079562 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 626/1268] 10.222.0.99 () {56 vars in 1143 bytes} [Thu Jul 14 15:26:19 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:26:20.085309 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928c2f3a-6481-4738-902b-45a9c207eb5d None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:26:20.188573 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928c2f3a-6481-4738-902b-45a9c207eb5d None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:26:20.223977 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e60178-600a-4f1a-a508-a6cb21b5a5e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:26:20.224774 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e60178-600a-4f1a-a508-a6cb21b5a5e8 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:26:20.225056 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e60178-600a-4f1a-a508-a6cb21b5a5e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:26:20.225330 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e60178-600a-4f1a-a508-a6cb21b5a5e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:26:20.236290 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e60178-600a-4f1a-a508-a6cb21b5a5e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:26:20.237752 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e60178-600a-4f1a-a508-a6cb21b5a5e8 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:26:20.255422 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-09e60178-600a-4f1a-a508-a6cb21b5a5e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:26:20.255422 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:26:20.255422 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:26:20.255422 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:26:20.255422 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:26:20.255422 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:26:20.255422 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:26:20.255422 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:26:20.255422 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:26:20.255422 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:26:20.255422 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:26:20.255422 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:26:20.255422 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:26:20.255422 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:26:20.255422 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:26:20.256609 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:26:20.256609 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:26:20.256609 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:26:20.256609 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:26:20.256609 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:26:20.256609 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:26:20.256609 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:26:20.256609 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:26:20.256609 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:26:20.256609 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:26:20.256609 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:26:20.256609 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:26:20.256609 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:26:20.256609 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:26:20.256609 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:26:20.256609 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:26:20.256609 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:26:20.257718 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:26:20.257718 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:26:20.257718 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:26:20.257718 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:26:20.257718 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 643/1269] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:26:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:26:20.262422 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aae985f-c78b-4769-b90f-88104414ccbd None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:26:20.363492 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aae985f-c78b-4769-b90f-88104414ccbd None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:26:20.399056 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad92cf59-448d-4f4f-951b-5bb85064bb9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:26:20.399821 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad92cf59-448d-4f4f-951b-5bb85064bb9a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:26:20.400111 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad92cf59-448d-4f4f-951b-5bb85064bb9a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:26:20.400396 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad92cf59-448d-4f4f-951b-5bb85064bb9a None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:26:20.401114 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad92cf59-448d-4f4f-951b-5bb85064bb9a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:26:20.401428 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad92cf59-448d-4f4f-951b-5bb85064bb9a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:26:20.402802 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad92cf59-448d-4f4f-951b-5bb85064bb9a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:26:20.414437 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 627/1270] 10.222.0.99 () {56 vars in 1109 bytes} [Thu Jul 14 15:26:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:26:20.420313 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e60178-600a-4f1a-a508-a6cb21b5a5e8 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:26:20.452681 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:26:20.527920 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e60178-600a-4f1a-a508-a6cb21b5a5e8 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:26:20.562636 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea269489-9bf7-40fc-9258-41e546e833be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:26:20.563428 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea269489-9bf7-40fc-9258-41e546e833be None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:26:20.563747 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea269489-9bf7-40fc-9258-41e546e833be None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:26:20.564030 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea269489-9bf7-40fc-9258-41e546e833be None admin] PATH_INFO: `/v3/projects/service` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:26:20.570696 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea269489-9bf7-40fc-9258-41e546e833be None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:26:20.572170 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea269489-9bf7-40fc-9258-41e546e833be None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:26:20.586607 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-ea269489-9bf7-40fc-9258-41e546e833be None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:26:20.586607 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:26:20.586607 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:26:20.586607 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:26:20.586607 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:26:20.586607 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:26:20.586607 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:26:20.586607 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:26:20.586607 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:26:20.586607 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:26:20.586607 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:26:20.586607 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:26:20.586607 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 15:26:20.586607 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 15:26:20.586607 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 15:26:20.587682 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 15:26:20.587682 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:26:20.587682 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:26:20.587682 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:26:20.587682 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:26:20.587682 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:26:20.587682 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:26:20.587682 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:26:20.587682 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:26:20.587682 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:26:20.587682 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:26:20.587682 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:26:20.587682 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:26:20.587682 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:26:20.587682 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:26:20.587682 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:26:20.587682 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:26:20.588993 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 15:26:20.588993 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 15:26:20.588993 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 15:26:20.588993 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 15:26:20.588993 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 15:26:20.588993 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 15:26:20.588993 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:26:20.588993 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:26:20.588993 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 644/1271] 10.222.0.99 () {56 vars in 1111 bytes} [Thu Jul 14 15:26:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:26:20.593742 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad92cf59-448d-4f4f-951b-5bb85064bb9a None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:26:20.653061 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:26:20.695263 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad92cf59-448d-4f4f-951b-5bb85064bb9a None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:26:20.735957 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d9305f-e8b6-4f55-a56a-86186b94f67d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:26:20.736363 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d9305f-e8b6-4f55-a56a-86186b94f67d None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:26:20.738031 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d9305f-e8b6-4f55-a56a-86186b94f67d None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:26:20.738031 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d9305f-e8b6-4f55-a56a-86186b94f67d None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:26:20.738031 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d9305f-e8b6-4f55-a56a-86186b94f67d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:26:20.738985 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d9305f-e8b6-4f55-a56a-86186b94f67d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:26:20.739902 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d9305f-e8b6-4f55-a56a-86186b94f67d None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:26:20.753112 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 628/1272] 10.222.0.99 () {56 vars in 1148 bytes} [Thu Jul 14 15:26:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:26:20.759266 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea269489-9bf7-40fc-9258-41e546e833be None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:26:20.847989 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:26:20.856885 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea269489-9bf7-40fc-9258-41e546e833be None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:26:20.893669 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8001c971-fe2b-4b2a-90eb-7d5a4be58035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:26:20.894562 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8001c971-fe2b-4b2a-90eb-7d5a4be58035 None admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:26:20.894852 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8001c971-fe2b-4b2a-90eb-7d5a4be58035 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:26:20.895135 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8001c971-fe2b-4b2a-90eb-7d5a4be58035 None admin] PATH_INFO: `/v3/projects/9048d1a6f00443ec92f11471babe5a56/users/dc157c485f794e968a60d8405c9310ae/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:26:20.897208 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8001c971-fe2b-4b2a-90eb-7d5a4be58035 None admin] RBAC: Authorizing `identity:create_grant(project_id=9048d1a6f00443ec92f11471babe5a56, user_id=dc157c485f794e968a60d8405c9310ae, role_id=d0f6ba1314f14d9f9c816bdb5d0c3aac)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:26:20.898652 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8001c971-fe2b-4b2a-90eb-7d5a4be58035 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:26:20.913102 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 645/1273] 10.222.0.99 () {58 vars in 1439 bytes} [Thu Jul 14 15:26:20 2022] PUT /identity/v3/projects/9048d1a6f00443ec92f11471babe5a56/users/dc157c485f794e968a60d8405c9310ae/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:26:23.004027 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e9080a-ec0b-4b8c-b6ef-42cf538e1364 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:26:23.004463 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e9080a-ec0b-4b8c-b6ef-42cf538e1364 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:26:23.004622 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e9080a-ec0b-4b8c-b6ef-42cf538e1364 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:26:23.006008 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 629/1274] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:26:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:26:23.011074 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6accf7bc-2fb7-4bc9-8848-40d25343a2e1 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:26:23.011443 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6accf7bc-2fb7-4bc9-8848-40d25343a2e1 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:26:23.011682 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6accf7bc-2fb7-4bc9-8848-40d25343a2e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:26:23.028010 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-6accf7bc-2fb7-4bc9-8848-40d25343a2e1 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:26:23.029359 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-6accf7bc-2fb7-4bc9-8848-40d25343a2e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:26:23.096470 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 646/1275] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:26:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 15:26:23.120551 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e9080a-ec0b-4b8c-b6ef-42cf538e1364 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:26:23.222410 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e9080a-ec0b-4b8c-b6ef-42cf538e1364 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:26:23.239728 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:26:23.256116 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949675c2-957b-4eb9-ab80-6daba53cc29b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:26:23.256835 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949675c2-957b-4eb9-ab80-6daba53cc29b None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:26:23.257118 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949675c2-957b-4eb9-ab80-6daba53cc29b None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:26:23.257395 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949675c2-957b-4eb9-ab80-6daba53cc29b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:26:23.267569 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949675c2-957b-4eb9-ab80-6daba53cc29b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:26:23.275054 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-949675c2-957b-4eb9-ab80-6daba53cc29b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:26:23.275054 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:26:23.275054 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:26:23.275054 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:26:23.275054 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:26:23.275054 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:26:23.275054 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:26:23.275054 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:26:23.275054 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:26:23.275054 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:26:23.275054 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:26:23.275054 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:26:23.275054 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:26:23.275054 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:26:23.275054 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:26:23.275054 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:26:23.276121 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:26:23.276121 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:26:23.276121 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:26:23.276121 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:26:23.276121 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:26:23.276121 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:26:23.276121 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:26:23.276121 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:26:23.276121 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:26:23.276121 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:26:23.276121 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:26:23.276121 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:26:23.276121 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:26:23.276121 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:26:23.276121 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:26:23.276121 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:26:23.277151 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:26:23.277151 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:26:23.277151 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:26:23.277151 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:26:23.277151 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:26:23.277151 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:26:23.277151 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:26:23.277151 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:26:23.277151 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:26:23.277151 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:26:23.277151 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:26:23.277151 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:26:23.277151 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 630/1276] 10.222.0.99 () {56 vars in 1096 bytes} [Thu Jul 14 15:26:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:26:23.283055 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6accf7bc-2fb7-4bc9-8848-40d25343a2e1 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:26:23.363443 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:26:23.382487 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6accf7bc-2fb7-4bc9-8848-40d25343a2e1 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:26:23.415738 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910a72fc-f759-4a75-871f-db5af1726fad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:26:23.416404 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910a72fc-f759-4a75-871f-db5af1726fad None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:26:23.416695 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910a72fc-f759-4a75-871f-db5af1726fad None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:26:23.416977 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910a72fc-f759-4a75-871f-db5af1726fad None admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:26:23.417731 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910a72fc-f759-4a75-871f-db5af1726fad None admin] RBAC: Adding query filter params (name=admin) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:26:23.418042 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910a72fc-f759-4a75-871f-db5af1726fad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:26:23.419413 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910a72fc-f759-4a75-871f-db5af1726fad None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:26:23.428353 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 647/1277] 10.222.0.99 () {56 vars in 1099 bytes} [Thu Jul 14 15:26:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:26:23.434182 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949675c2-957b-4eb9-ab80-6daba53cc29b None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:26:23.537998 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949675c2-957b-4eb9-ab80-6daba53cc29b None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:26:23.570720 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0055ac80-e351-4cc3-8aa9-6f065fd30a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:26:23.571508 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0055ac80-e351-4cc3-8aa9-6f065fd30a33 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:26:23.571825 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0055ac80-e351-4cc3-8aa9-6f065fd30a33 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:26:23.572107 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0055ac80-e351-4cc3-8aa9-6f065fd30a33 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:26:23.583684 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0055ac80-e351-4cc3-8aa9-6f065fd30a33 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:26:23.585126 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0055ac80-e351-4cc3-8aa9-6f065fd30a33 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:26:23.602584 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-0055ac80-e351-4cc3-8aa9-6f065fd30a33 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:26:23.602584 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:26:23.602584 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:26:23.602584 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:26:23.602584 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:26:23.602584 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:26:23.602584 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:26:23.602584 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:26:23.602584 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:26:23.602584 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:26:23.602584 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:26:23.602584 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:26:23.602584 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:26:23.602584 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:26:23.602584 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:26:23.603681 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:26:23.603681 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:26:23.603681 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:26:23.603681 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:26:23.603681 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:26:23.603681 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:26:23.603681 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:26:23.603681 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:26:23.603681 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:26:23.603681 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:26:23.603681 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:26:23.603681 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:26:23.603681 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:26:23.603681 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:26:23.603681 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:26:23.603681 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:26:23.603681 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:26:23.604978 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:26:23.604978 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:26:23.604978 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:26:23.604978 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:26:23.604978 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 631/1278] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:26:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 15:26:23.610288 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910a72fc-f759-4a75-871f-db5af1726fad None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:26:23.711669 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910a72fc-f759-4a75-871f-db5af1726fad None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:26:23.751034 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd893eb-8581-4eb5-939e-f781edaeb454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:26:23.751781 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd893eb-8581-4eb5-939e-f781edaeb454 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:26:23.752022 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd893eb-8581-4eb5-939e-f781edaeb454 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:26:23.752319 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd893eb-8581-4eb5-939e-f781edaeb454 None admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:26:23.753029 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd893eb-8581-4eb5-939e-f781edaeb454 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:26:23.753334 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd893eb-8581-4eb5-939e-f781edaeb454 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:26:23.754701 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd893eb-8581-4eb5-939e-f781edaeb454 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:26:23.767188 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 648/1279] 10.222.0.99 () {56 vars in 1109 bytes} [Thu Jul 14 15:26:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:26:23.773162 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0055ac80-e351-4cc3-8aa9-6f065fd30a33 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:26:23.808463 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:26:23.876737 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0055ac80-e351-4cc3-8aa9-6f065fd30a33 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:26:23.910707 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36917aa-09c6-4aa0-81a5-bea28a503007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:26:23.911498 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36917aa-09c6-4aa0-81a5-bea28a503007 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:26:23.911812 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36917aa-09c6-4aa0-81a5-bea28a503007 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:26:23.912092 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36917aa-09c6-4aa0-81a5-bea28a503007 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:26:23.919422 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36917aa-09c6-4aa0-81a5-bea28a503007 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:26:23.920973 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36917aa-09c6-4aa0-81a5-bea28a503007 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:26:23.935083 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-b36917aa-09c6-4aa0-81a5-bea28a503007 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 14 15:26:23.935083 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:26:23.935083 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:26:23.935083 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:26:23.935083 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:26:23.935083 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:26:23.935083 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:26:23.935083 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:26:23.935083 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:26:23.935083 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:26:23.935083 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:26:23.935083 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:26:23.935083 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 15:26:23.935083 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 15:26:23.935083 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 15:26:23.935083 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 15:26:23.936686 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:26:23.936686 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:26:23.936686 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 15:26:23.936686 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:26:23.936686 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 15:26:23.936686 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:26:23.936686 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:26:23.936686 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:26:23.936686 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:26:23.936686 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:26:23.936686 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:26:23.936686 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:26:23.936686 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:26:23.936686 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:26:23.936686 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:26:23.936686 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:26:23.936686 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:26:23.936686 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:26:23.937780 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:26:23.937780 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:26:23.937780 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 15:26:23.937780 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 15:26:23.937780 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 15:26:23.937780 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 15:26:23.937780 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 15:26:23.937780 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 15:26:23.937780 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 14 15:26:23.937780 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:26:23.937780 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 632/1280] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:26:23 2022] GET /identity/v3/users/placement => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:26:23.942935 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd893eb-8581-4eb5-939e-f781edaeb454 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:26:23.946711 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:26:24.043747 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd893eb-8581-4eb5-939e-f781edaeb454 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:26:24.077781 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0becd473-0f3c-4787-8ae2-f231b37eff91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:26:24.078376 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0becd473-0f3c-4787-8ae2-f231b37eff91 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:26:24.078663 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0becd473-0f3c-4787-8ae2-f231b37eff91 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:26:24.078940 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0becd473-0f3c-4787-8ae2-f231b37eff91 None admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:26:24.079805 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0becd473-0f3c-4787-8ae2-f231b37eff91 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:26:24.080116 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0becd473-0f3c-4787-8ae2-f231b37eff91 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:26:24.081530 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0becd473-0f3c-4787-8ae2-f231b37eff91 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:26:24.098422 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 649/1281] 10.222.0.99 () {56 vars in 1143 bytes} [Thu Jul 14 15:26:23 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:26:24.104290 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36917aa-09c6-4aa0-81a5-bea28a503007 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:26:24.195865 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:26:24.206523 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36917aa-09c6-4aa0-81a5-bea28a503007 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:26:24.238561 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d187df-cc4c-435e-a906-6ce2035eb272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:26:24.239199 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d187df-cc4c-435e-a906-6ce2035eb272 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:26:24.239484 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d187df-cc4c-435e-a906-6ce2035eb272 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:26:24.239795 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d187df-cc4c-435e-a906-6ce2035eb272 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:26:24.304812 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d187df-cc4c-435e-a906-6ce2035eb272 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:26:24.306243 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d187df-cc4c-435e-a906-6ce2035eb272 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:26:24.393480 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:26:24.403478 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 633/1282] 10.222.0.99 () {58 vars in 1282 bytes} [Thu Jul 14 15:26:24 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 15:26:24.409557 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0becd473-0f3c-4787-8ae2-f231b37eff91 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:26:24.470552 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:26:24.509689 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0becd473-0f3c-4787-8ae2-f231b37eff91 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:26:24.542850 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0fbfac-63b6-4dd5-aa59-9991b27b642e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:26:24.543664 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0fbfac-63b6-4dd5-aa59-9991b27b642e None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:26:24.543956 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0fbfac-63b6-4dd5-aa59-9991b27b642e None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:26:24.544240 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0fbfac-63b6-4dd5-aa59-9991b27b642e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:26:24.554799 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0fbfac-63b6-4dd5-aa59-9991b27b642e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:26:24.556283 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0fbfac-63b6-4dd5-aa59-9991b27b642e None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:26:24.574887 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-ed0fbfac-63b6-4dd5-aa59-9991b27b642e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:26:24.574887 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:26:24.574887 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:26:24.574887 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:26:24.574887 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:26:24.574887 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:26:24.574887 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:26:24.574887 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:26:24.574887 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:26:24.574887 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:26:24.574887 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:26:24.574887 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:26:24.574887 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:26:24.574887 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:26:24.574887 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:26:24.576060 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:26:24.576060 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:26:24.576060 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:26:24.576060 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:26:24.576060 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:26:24.576060 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:26:24.576060 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:26:24.576060 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:26:24.576060 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:26:24.576060 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:26:24.576060 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:26:24.576060 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:26:24.576060 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:26:24.576060 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:26:24.576060 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:26:24.576060 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:26:24.576060 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:26:24.577590 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:26:24.577590 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:26:24.577590 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:26:24.577590 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:26:24.577590 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 650/1283] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:26:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:26:24.582071 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d187df-cc4c-435e-a906-6ce2035eb272 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:26:24.679180 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d187df-cc4c-435e-a906-6ce2035eb272 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:26:24.714523 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12694e66-ecee-46d4-ab61-00b7e79c7985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:26:24.715275 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12694e66-ecee-46d4-ab61-00b7e79c7985 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:26:24.715468 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12694e66-ecee-46d4-ab61-00b7e79c7985 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:26:24.715768 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12694e66-ecee-46d4-ab61-00b7e79c7985 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:26:24.717000 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12694e66-ecee-46d4-ab61-00b7e79c7985 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:26:24.717314 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12694e66-ecee-46d4-ab61-00b7e79c7985 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:26:24.718794 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12694e66-ecee-46d4-ab61-00b7e79c7985 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:26:24.730029 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 634/1284] 10.222.0.99 () {56 vars in 1109 bytes} [Thu Jul 14 15:26:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:26:24.736443 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0fbfac-63b6-4dd5-aa59-9991b27b642e None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:26:24.833708 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0fbfac-63b6-4dd5-aa59-9991b27b642e None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:26:24.847945 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:26:24.869532 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6010f5-0b51-45eb-b009-7fc19a4b061c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:26:24.870314 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6010f5-0b51-45eb-b009-7fc19a4b061c None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:26:24.870596 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6010f5-0b51-45eb-b009-7fc19a4b061c None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:26:24.870870 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6010f5-0b51-45eb-b009-7fc19a4b061c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:26:24.877466 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6010f5-0b51-45eb-b009-7fc19a4b061c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:26:24.878893 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6010f5-0b51-45eb-b009-7fc19a4b061c None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:26:24.892678 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-5b6010f5-0b51-45eb-b009-7fc19a4b061c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:26:24.892678 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:26:24.892678 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:26:24.892678 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:26:24.892678 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:26:24.892678 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:26:24.892678 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:26:24.892678 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:26:24.892678 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:26:24.892678 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:26:24.892678 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:26:24.892678 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:26:24.892678 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 15:26:24.892678 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 15:26:24.892678 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 15:26:24.893716 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 15:26:24.893716 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:26:24.893716 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:26:24.893716 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:26:24.893716 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:26:24.893716 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:26:24.893716 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:26:24.893716 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:26:24.893716 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 15:26:24.893716 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:26:24.893716 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:26:24.893716 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:26:24.893716 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:26:24.893716 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:26:24.893716 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:26:24.893716 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:26:24.893716 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:26:24.894663 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 15:26:24.894663 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 15:26:24.894663 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 15:26:24.894663 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 15:26:24.894663 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 15:26:24.894663 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 15:26:24.894663 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:26:24.894663 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 15:26:24.894663 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 651/1285] 10.222.0.99 () {56 vars in 1111 bytes} [Thu Jul 14 15:26:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:26:24.899932 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12694e66-ecee-46d4-ab61-00b7e79c7985 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:26:24.950030 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:26:25.001290 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12694e66-ecee-46d4-ab61-00b7e79c7985 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:26:25.035654 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce61cd65-6571-4fac-9980-4c1f255dc98d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:26:25.036316 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce61cd65-6571-4fac-9980-4c1f255dc98d None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:26:25.036606 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce61cd65-6571-4fac-9980-4c1f255dc98d None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:26:25.036887 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce61cd65-6571-4fac-9980-4c1f255dc98d None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:26:25.037673 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce61cd65-6571-4fac-9980-4c1f255dc98d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:26:25.037988 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce61cd65-6571-4fac-9980-4c1f255dc98d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:26:25.039418 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce61cd65-6571-4fac-9980-4c1f255dc98d None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:26:25.052052 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 635/1286] 10.222.0.99 () {56 vars in 1148 bytes} [Thu Jul 14 15:26:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:26:25.058281 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6010f5-0b51-45eb-b009-7fc19a4b061c None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:26:25.157190 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6010f5-0b51-45eb-b009-7fc19a4b061c None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:26:25.191345 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474d383e-26c0-4625-8122-0712719d8606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:26:25.192024 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474d383e-26c0-4625-8122-0712719d8606 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:26:25.192305 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474d383e-26c0-4625-8122-0712719d8606 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:26:25.192583 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474d383e-26c0-4625-8122-0712719d8606 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:26:25.193417 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474d383e-26c0-4625-8122-0712719d8606 None admin] RBAC: Adding query filter params (role.id=d0f6ba1314f14d9f9c816bdb5d0c3aac, scope.project.id=9048d1a6f00443ec92f11471babe5a56, user.id=dc157c485f794e968a60d8405c9310ae) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:26:25.193726 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474d383e-26c0-4625-8122-0712719d8606 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:26:25.195157 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474d383e-26c0-4625-8122-0712719d8606 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:26:25.201601 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 652/1287] 10.222.0.99 () {56 vars in 1374 bytes} [Thu Jul 14 15:26:25 2022] GET /identity/v3/role_assignments?user.id=dc157c485f794e968a60d8405c9310ae&scope.project.id=9048d1a6f00443ec92f11471babe5a56&role.id=d0f6ba1314f14d9f9c816bdb5d0c3aac => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:26:27.304496 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d359b61-f6a6-4a46-bb69-dd7a39cfe3d7 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:26:27.305021 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d359b61-f6a6-4a46-bb69-dd7a39cfe3d7 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:26:27.305021 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d359b61-f6a6-4a46-bb69-dd7a39cfe3d7 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:26:27.306348 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 636/1288] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:26:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:26:27.311905 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df94b8cc-0b2d-48ee-b3fd-64d5270f6ed9 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:26:27.312345 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df94b8cc-0b2d-48ee-b3fd-64d5270f6ed9 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:26:27.312477 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df94b8cc-0b2d-48ee-b3fd-64d5270f6ed9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:26:27.329781 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-df94b8cc-0b2d-48ee-b3fd-64d5270f6ed9 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:26:27.330998 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-df94b8cc-0b2d-48ee-b3fd-64d5270f6ed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:26:27.364102 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:26:27.402731 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 653/1289] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:26:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 15:26:27.424910 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d359b61-f6a6-4a46-bb69-dd7a39cfe3d7 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:26:27.547678 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d359b61-f6a6-4a46-bb69-dd7a39cfe3d7 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:26:27.558121 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:26:27.582247 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b960379-6bab-483e-b5ef-aceab889ae26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:26:27.583031 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b960379-6bab-483e-b5ef-aceab889ae26 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:26:27.583319 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b960379-6bab-483e-b5ef-aceab889ae26 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:26:27.583623 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b960379-6bab-483e-b5ef-aceab889ae26 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:26:27.584266 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b960379-6bab-483e-b5ef-aceab889ae26 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:26:27.585751 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b960379-6bab-483e-b5ef-aceab889ae26 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:26:27.597819 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-8b960379-6bab-483e-b5ef-aceab889ae26 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 14 15:26:27.597819 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:26:27.597819 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 15:26:27.597819 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 15:26:27.597819 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 15:26:27.597819 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 15:26:27.597819 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 15:26:27.597819 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 15:26:27.597819 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 14 15:26:27.597819 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:26:27.597819 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 15:26:27.597819 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:26:27.597819 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:26:27.597819 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:26:27.597819 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:26:27.597819 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:26:27.597819 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:26:27.599031 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:26:27.599031 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:26:27.599031 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:26:27.599031 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:26:27.599031 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:26:27.599031 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:26:27.599031 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 15:26:27.599031 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 15:26:27.599031 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 15:26:27.599031 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 15:26:27.599031 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:26:27.599031 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:26:27.599031 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:26:27.599031 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:26:27.599031 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:26:27.599031 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:26:27.599980 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:26:27.599980 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:26:27.599980 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:26:27.599980 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:26:27.599980 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:26:27.599980 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:26:27.599980 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:26:27.599980 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:26:27.599980 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:26:27.599980 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:26:27.599980 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 15:26:27.599980 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 15:26:27.599980 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 14 15:26:27.599980 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:26:27.599980 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 637/1290] 10.222.0.99 () {56 vars in 1117 bytes} [Thu Jul 14 15:26:27 2022] GET /identity/v3/services/placement => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:26:27.605203 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df94b8cc-0b2d-48ee-b3fd-64d5270f6ed9 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:26:27.709975 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df94b8cc-0b2d-48ee-b3fd-64d5270f6ed9 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:26:27.756973 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de49242f-c075-468e-9a7b-0ac467e5a66f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:26:27.757804 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de49242f-c075-468e-9a7b-0ac467e5a66f None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:26:27.758218 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de49242f-c075-468e-9a7b-0ac467e5a66f None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:26:27.758610 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de49242f-c075-468e-9a7b-0ac467e5a66f None admin] PATH_INFO: `/v3/services` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:26:27.759508 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de49242f-c075-468e-9a7b-0ac467e5a66f None admin] RBAC: Adding query filter params (name=placement) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:26:27.759853 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de49242f-c075-468e-9a7b-0ac467e5a66f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:26:27.761435 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de49242f-c075-468e-9a7b-0ac467e5a66f None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:26:27.768635 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 654/1291] 10.222.0.99 () {56 vars in 1116 bytes} [Thu Jul 14 15:26:27 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:26:27.776169 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b960379-6bab-483e-b5ef-aceab889ae26 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:26:27.875878 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b960379-6bab-483e-b5ef-aceab889ae26 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:26:27.908614 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcb8794-c1d1-4af3-a532-165910aa3ee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:26:27.909278 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcb8794-c1d1-4af3-a532-165910aa3ee7 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:26:27.909543 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcb8794-c1d1-4af3-a532-165910aa3ee7 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:26:27.909822 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcb8794-c1d1-4af3-a532-165910aa3ee7 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:26:27.910521 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcb8794-c1d1-4af3-a532-165910aa3ee7 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:26:27.910832 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcb8794-c1d1-4af3-a532-165910aa3ee7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:26:27.912220 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcb8794-c1d1-4af3-a532-165910aa3ee7 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:26:27.916600 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 638/1292] 10.222.0.99 () {56 vars in 1116 bytes} [Thu Jul 14 15:26:27 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:26:29.948037 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48751fef-0dbe-442b-9379-8d3d7b2b6235 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:26:29.948572 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48751fef-0dbe-442b-9379-8d3d7b2b6235 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:26:29.948667 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48751fef-0dbe-442b-9379-8d3d7b2b6235 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:26:29.949951 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 655/1293] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:26:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:26:29.955627 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2674ed-7923-4a34-b103-7f8be16e486b None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:26:29.956209 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2674ed-7923-4a34-b103-7f8be16e486b None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:26:29.956209 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2674ed-7923-4a34-b103-7f8be16e486b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:26:29.973422 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-df2674ed-7923-4a34-b103-7f8be16e486b None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:26:29.974595 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-df2674ed-7923-4a34-b103-7f8be16e486b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:26:30.004366 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:26:30.041937 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 639/1294] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:26:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 15:26:30.065777 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48751fef-0dbe-442b-9379-8d3d7b2b6235 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:26:30.096756 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:26:30.165723 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48751fef-0dbe-442b-9379-8d3d7b2b6235 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:26:30.202242 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f434f4e9-bd3f-4e7d-b059-90800037d351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:26:30.202835 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f434f4e9-bd3f-4e7d-b059-90800037d351 None admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:26:30.203125 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f434f4e9-bd3f-4e7d-b059-90800037d351 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:26:30.203408 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f434f4e9-bd3f-4e7d-b059-90800037d351 None admin] PATH_INFO: `/v3/services` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:26:30.204170 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f434f4e9-bd3f-4e7d-b059-90800037d351 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:26:30.205571 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f434f4e9-bd3f-4e7d-b059-90800037d351 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:26:30.219155 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 656/1295] 10.222.0.99 () {60 vars in 1141 bytes} [Thu Jul 14 15:26:30 2022] POST /identity/v3/services => generated 246 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:26:32.318253 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360dea47-8e59-40f4-b099-c074aa09cab9 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:26:32.318760 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360dea47-8e59-40f4-b099-c074aa09cab9 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:26:32.318760 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360dea47-8e59-40f4-b099-c074aa09cab9 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:26:32.320142 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 640/1296] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:26:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:26:32.325172 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5f49a7-a811-4537-b7ea-0875c49ccd90 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:26:32.325608 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5f49a7-a811-4537-b7ea-0875c49ccd90 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:26:32.325728 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5f49a7-a811-4537-b7ea-0875c49ccd90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:26:32.343940 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-0f5f49a7-a811-4537-b7ea-0875c49ccd90 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:26:32.345144 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-0f5f49a7-a811-4537-b7ea-0875c49ccd90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:26:32.427573 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 657/1297] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:26:32 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 15:26:32.452121 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360dea47-8e59-40f4-b099-c074aa09cab9 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:26:32.552836 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360dea47-8e59-40f4-b099-c074aa09cab9 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:26:32.586314 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75beef41-14aa-4d62-996f-4c588a81da86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:26:32.587037 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75beef41-14aa-4d62-996f-4c588a81da86 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:26:32.587324 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75beef41-14aa-4d62-996f-4c588a81da86 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:26:32.587630 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75beef41-14aa-4d62-996f-4c588a81da86 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:26:32.588285 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75beef41-14aa-4d62-996f-4c588a81da86 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:26:32.589640 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75beef41-14aa-4d62-996f-4c588a81da86 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:26:32.601609 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-75beef41-14aa-4d62-996f-4c588a81da86 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 14 15:26:32.601609 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:26:32.601609 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 15:26:32.601609 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 15:26:32.601609 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 15:26:32.601609 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 15:26:32.601609 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 15:26:32.601609 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 15:26:32.601609 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 14 15:26:32.601609 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:26:32.601609 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 15:26:32.601609 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:26:32.601609 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:26:32.601609 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:26:32.601609 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:26:32.601609 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:26:32.601609 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:26:32.602618 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:26:32.602618 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:26:32.602618 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:26:32.602618 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:26:32.602618 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:26:32.602618 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:26:32.602618 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 15:26:32.602618 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 15:26:32.602618 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 15:26:32.602618 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 15:26:32.602618 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:26:32.602618 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:26:32.602618 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:26:32.602618 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:26:32.602618 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:26:32.602618 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:26:32.603519 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:26:32.603519 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:26:32.603519 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:26:32.603519 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:26:32.603519 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:26:32.603519 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:26:32.603519 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:26:32.603519 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:26:32.603519 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:26:32.603519 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:26:32.603519 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 15:26:32.603519 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 15:26:32.603519 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 14 15:26:32.603519 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:26:32.603519 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 641/1298] 10.222.0.99 () {56 vars in 1117 bytes} [Thu Jul 14 15:26:32 2022] GET /identity/v3/services/placement => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:26:32.609732 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5f49a7-a811-4537-b7ea-0875c49ccd90 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:26:32.628642 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:26:32.713130 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5f49a7-a811-4537-b7ea-0875c49ccd90 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:26:32.755368 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac8768c-ab6a-464a-bcf5-bc0bd532e177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:26:32.756072 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac8768c-ab6a-464a-bcf5-bc0bd532e177 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:26:32.756363 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac8768c-ab6a-464a-bcf5-bc0bd532e177 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:26:32.756645 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac8768c-ab6a-464a-bcf5-bc0bd532e177 None admin] PATH_INFO: `/v3/services` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:26:32.757364 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac8768c-ab6a-464a-bcf5-bc0bd532e177 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:26:32.757676 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac8768c-ab6a-464a-bcf5-bc0bd532e177 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:26:32.759086 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac8768c-ab6a-464a-bcf5-bc0bd532e177 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:26:32.764443 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 658/1299] 10.222.0.99 () {56 vars in 1116 bytes} [Thu Jul 14 15:26:32 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:26:32.770430 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75beef41-14aa-4d62-996f-4c588a81da86 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:26:32.802702 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:26:32.877407 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75beef41-14aa-4d62-996f-4c588a81da86 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:26:32.915624 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b31f67-851f-4c52-bd5b-d5ce2201398c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:26:32.916269 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b31f67-851f-4c52-bd5b-d5ce2201398c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:26:32.916557 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b31f67-851f-4c52-bd5b-d5ce2201398c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:26:32.916835 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b31f67-851f-4c52-bd5b-d5ce2201398c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:26:32.917625 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b31f67-851f-4c52-bd5b-d5ce2201398c None admin] RBAC: Adding query filter params (interface=public, service_id=bd573732e4ac4f1d9fffb46560f3e250, region_id=RegionOne) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:26:32.917943 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b31f67-851f-4c52-bd5b-d5ce2201398c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:26:32.919309 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b31f67-851f-4c52-bd5b-d5ce2201398c None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:26:32.925341 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 642/1300] 10.222.0.99 () {56 vars in 1251 bytes} [Thu Jul 14 15:26:32 2022] GET /identity/v3/endpoints?service_id=bd573732e4ac4f1d9fffb46560f3e250&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:26:32.930941 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac8768c-ab6a-464a-bcf5-bc0bd532e177 None admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:26:33.013206 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:26:33.033139 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac8768c-ab6a-464a-bcf5-bc0bd532e177 None admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:26:33.066424 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e94e1e5-cd6d-4eb9-a035-bc6a8443d8c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:26:33.067084 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e94e1e5-cd6d-4eb9-a035-bc6a8443d8c4 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:26:33.067372 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e94e1e5-cd6d-4eb9-a035-bc6a8443d8c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:26:33.067678 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e94e1e5-cd6d-4eb9-a035-bc6a8443d8c4 None admin] PATH_INFO: `/v3/services` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:26:33.068388 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e94e1e5-cd6d-4eb9-a035-bc6a8443d8c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:26:33.069755 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e94e1e5-cd6d-4eb9-a035-bc6a8443d8c4 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:26:33.076558 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 659/1301] 10.222.0.99 () {56 vars in 1087 bytes} [Thu Jul 14 15:26:32 2022] GET /identity/v3/services => generated 2191 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:26:35.123669 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9697da-aa0e-4335-ad7b-f94a875c6149 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:26:35.124196 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9697da-aa0e-4335-ad7b-f94a875c6149 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:26:35.124313 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9697da-aa0e-4335-ad7b-f94a875c6149 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:26:35.125617 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 643/1302] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:26:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:26:35.130745 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c36bf7-bd48-4cdb-9600-d37ae7fc92a3 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:26:35.131161 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c36bf7-bd48-4cdb-9600-d37ae7fc92a3 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:26:35.131319 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c36bf7-bd48-4cdb-9600-d37ae7fc92a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:26:35.148456 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-a7c36bf7-bd48-4cdb-9600-d37ae7fc92a3 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:26:35.149640 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-a7c36bf7-bd48-4cdb-9600-d37ae7fc92a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:26:35.218189 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 660/1303] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:26:35 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 15:26:35.241542 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9697da-aa0e-4335-ad7b-f94a875c6149 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:26:35.338537 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:26:35.345936 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9697da-aa0e-4335-ad7b-f94a875c6149 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:26:35.379372 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431a51a7-2d6f-40e5-bc7a-6d3ecd541fb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:26:35.380192 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431a51a7-2d6f-40e5-bc7a-6d3ecd541fb2 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:26:35.380481 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431a51a7-2d6f-40e5-bc7a-6d3ecd541fb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:26:35.380763 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431a51a7-2d6f-40e5-bc7a-6d3ecd541fb2 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:26:35.381419 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431a51a7-2d6f-40e5-bc7a-6d3ecd541fb2 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:26:35.382777 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431a51a7-2d6f-40e5-bc7a-6d3ecd541fb2 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:26:35.395750 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-431a51a7-2d6f-40e5-bc7a-6d3ecd541fb2 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 14 15:26:35.395750 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:26:35.395750 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 15:26:35.395750 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 15:26:35.395750 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 15:26:35.395750 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 15:26:35.395750 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 15:26:35.395750 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 15:26:35.395750 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 14 15:26:35.395750 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:26:35.395750 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 15:26:35.395750 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:26:35.395750 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:26:35.395750 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:26:35.395750 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:26:35.395750 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:26:35.395750 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:26:35.396991 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:26:35.396991 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:26:35.396991 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:26:35.396991 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:26:35.396991 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:26:35.396991 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:26:35.396991 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 15:26:35.396991 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 15:26:35.396991 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 15:26:35.396991 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 15:26:35.396991 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:26:35.396991 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:26:35.396991 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:26:35.396991 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:26:35.396991 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:26:35.396991 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:26:35.397940 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:26:35.397940 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 14 15:26:35.397940 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:26:35.397940 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:26:35.397940 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:26:35.397940 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:26:35.397940 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:26:35.397940 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:26:35.397940 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:26:35.397940 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:26:35.397940 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 15:26:35.397940 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 15:26:35.397940 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 14 15:26:35.397940 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 14 15:26:35.397940 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 644/1304] 10.222.0.99 () {56 vars in 1117 bytes} [Thu Jul 14 15:26:35 2022] GET /identity/v3/services/placement => generated 90 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:26:35.405409 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c36bf7-bd48-4cdb-9600-d37ae7fc92a3 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:26:35.481742 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:26:35.505231 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c36bf7-bd48-4cdb-9600-d37ae7fc92a3 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:26:35.539299 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61164591-d7aa-465a-b827-3013612f1058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:26:35.539993 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61164591-d7aa-465a-b827-3013612f1058 None admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:26:35.540285 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61164591-d7aa-465a-b827-3013612f1058 None admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:26:35.540571 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61164591-d7aa-465a-b827-3013612f1058 None admin] PATH_INFO: `/v3/services` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:26:35.541286 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61164591-d7aa-465a-b827-3013612f1058 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:26:35.541599 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61164591-d7aa-465a-b827-3013612f1058 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:26:35.542982 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61164591-d7aa-465a-b827-3013612f1058 None admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:26:35.548757 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 661/1305] 10.222.0.99 () {56 vars in 1116 bytes} [Thu Jul 14 15:26:35 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:26:35.554762 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431a51a7-2d6f-40e5-bc7a-6d3ecd541fb2 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:26:35.656645 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431a51a7-2d6f-40e5-bc7a-6d3ecd541fb2 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:26:35.689076 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8743344-577f-4c15-a38a-d064265aaa93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:26:35.689735 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8743344-577f-4c15-a38a-d064265aaa93 None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:26:35.690028 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8743344-577f-4c15-a38a-d064265aaa93 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:26:35.690313 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8743344-577f-4c15-a38a-d064265aaa93 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:26:35.691060 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8743344-577f-4c15-a38a-d064265aaa93 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:26:35.692558 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8743344-577f-4c15-a38a-d064265aaa93 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:26:35.713259 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 645/1306] 10.222.0.99 () {60 vars in 1144 bytes} [Thu Jul 14 15:26:35 2022] POST /identity/v3/endpoints => generated 331 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:26:45.545691 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=30 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:26:45.707425 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:27:41.575104 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf394d5-ed40-483a-9e77-f497ff612c06 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:27:41.575525 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf394d5-ed40-483a-9e77-f497ff612c06 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:27:41.576042 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf394d5-ed40-483a-9e77-f497ff612c06 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:27:41.579395 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 662/1307] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:27:41 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:27:41.584658 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1e1aee-7d4f-4c0e-8209-2b4de5ed3d30 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:27:41.585297 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1e1aee-7d4f-4c0e-8209-2b4de5ed3d30 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:27:41.585297 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1e1aee-7d4f-4c0e-8209-2b4de5ed3d30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:27:41.744486 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-cf1e1aee-7d4f-4c0e-8209-2b4de5ed3d30 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:27:41.745896 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-cf1e1aee-7d4f-4c0e-8209-2b4de5ed3d30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:27:41.881616 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 646/1308] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:27:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 15:27:41.934102 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f305f0a-b36c-4005-8b1b-f7429f8821e2 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:27:41.934392 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f305f0a-b36c-4005-8b1b-f7429f8821e2 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:27:41.934677 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f305f0a-b36c-4005-8b1b-f7429f8821e2 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:27:41.937375 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 663/1309] 10.222.0.99 () {54 vars in 964 bytes} [Thu Jul 14 15:27:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:27:41.943932 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102dde22-1fef-455f-91ac-794b775ea10e None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:27:41.944377 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102dde22-1fef-455f-91ac-794b775ea10e None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:27:41.944764 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102dde22-1fef-455f-91ac-794b775ea10e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:27:42.043062 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-102dde22-1fef-455f-91ac-794b775ea10e None None] MFA Rules not processed for user `0d0261759f9743b1a13a760ca9159305`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:27:42.044323 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-102dde22-1fef-455f-91ac-794b775ea10e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:27:42.069796 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:27:42.136154 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 647/1310] 10.222.0.99 () {58 vars in 1061 bytes} [Thu Jul 14 15:27:41 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:27:42.146217 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f305f0a-b36c-4005-8b1b-f7429f8821e2 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:27:42.148041 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-9f305f0a-b36c-4005-8b1b-f7429f8821e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:27:42.324360 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f305f0a-b36c-4005-8b1b-f7429f8821e2 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:27:42.423494 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d1096d-7b61-487a-9931-2d29e85b18c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:27:42.424236 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d1096d-7b61-487a-9931-2d29e85b18c7 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:27:42.424506 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d1096d-7b61-487a-9931-2d29e85b18c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:27:42.424752 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d1096d-7b61-487a-9931-2d29e85b18c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:27:42.426498 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-e1d1096d-7b61-487a-9931-2d29e85b18c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:27:42.502910 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:27:42.589331 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d1096d-7b61-487a-9931-2d29e85b18c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:27:42.591118 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d1096d-7b61-487a-9931-2d29e85b18c7 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:27:42.614020 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 664/1311] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:27:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 15:27:45.077042 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a8ca9a-aa73-46d2-8c0c-707b81c56820 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:27:45.077647 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a8ca9a-aa73-46d2-8c0c-707b81c56820 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:27:45.077647 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a8ca9a-aa73-46d2-8c0c-707b81c56820 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:27:45.079059 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 648/1312] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:27:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:27:45.085241 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b16e28-0136-484f-95a5-949482bd6f99 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:27:45.085696 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b16e28-0136-484f-95a5-949482bd6f99 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:27:45.085807 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b16e28-0136-484f-95a5-949482bd6f99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:27:45.157857 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-c8b16e28-0136-484f-95a5-949482bd6f99 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:27:45.159089 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-c8b16e28-0136-484f-95a5-949482bd6f99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:27:45.261358 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 665/1313] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:27:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:27:45.304339 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a8ca9a-aa73-46d2-8c0c-707b81c56820 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:27:45.305884 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a7a8ca9a-aa73-46d2-8c0c-707b81c56820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:27:45.417449 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a8ca9a-aa73-46d2-8c0c-707b81c56820 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:27:45.491819 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:27:45.494531 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e7db30-bc38-4cc0-9a3c-0c27b7f2abb6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:27:45.495406 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e7db30-bc38-4cc0-9a3c-0c27b7f2abb6 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:27:45.495829 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e7db30-bc38-4cc0-9a3c-0c27b7f2abb6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:27:45.496342 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e7db30-bc38-4cc0-9a3c-0c27b7f2abb6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:27:45.498129 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a4e7db30-bc38-4cc0-9a3c-0c27b7f2abb6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:27:45.613439 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e7db30-bc38-4cc0-9a3c-0c27b7f2abb6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:27:45.614907 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e7db30-bc38-4cc0-9a3c-0c27b7f2abb6 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:27:45.633946 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 649/1314] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:27:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:27:48.127874 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76f033f-7c80-445b-86bc-236bc60784b8 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:27:48.128458 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76f033f-7c80-445b-86bc-236bc60784b8 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:27:48.128458 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76f033f-7c80-445b-86bc-236bc60784b8 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:27:48.129956 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 666/1315] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:27:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:27:48.135511 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf3374d-677c-4ae5-969a-02a97c2ceff5 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:27:48.135976 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf3374d-677c-4ae5-969a-02a97c2ceff5 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:27:48.136089 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf3374d-677c-4ae5-969a-02a97c2ceff5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:27:48.208141 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-6cf3374d-677c-4ae5-969a-02a97c2ceff5 None None] MFA Rules not processed for user `7f5177fcb1524b918c6662c934105643`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:27:48.209392 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-6cf3374d-677c-4ae5-969a-02a97c2ceff5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:27:48.264788 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:27:48.314344 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 650/1316] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:27:48 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:27:48.361191 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76f033f-7c80-445b-86bc-236bc60784b8 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:27:48.362866 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-c76f033f-7c80-445b-86bc-236bc60784b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:27:48.376977 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:27:48.474402 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76f033f-7c80-445b-86bc-236bc60784b8 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:27:48.557741 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ad4696-32c8-4aa6-9456-320711d24d43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:27:48.558426 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ad4696-32c8-4aa6-9456-320711d24d43 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:27:48.558703 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ad4696-32c8-4aa6-9456-320711d24d43 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:27:48.558982 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ad4696-32c8-4aa6-9456-320711d24d43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:27:48.561298 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-45ad4696-32c8-4aa6-9456-320711d24d43 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:27:48.628578 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:27:48.678490 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ad4696-32c8-4aa6-9456-320711d24d43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:27:48.679772 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ad4696-32c8-4aa6-9456-320711d24d43 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:27:48.703809 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 667/1317] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:27:48 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:27:51.408619 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1d46ab-b254-4283-9d3c-384ffd442da8 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:27:51.409353 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1d46ab-b254-4283-9d3c-384ffd442da8 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:27:51.409353 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1d46ab-b254-4283-9d3c-384ffd442da8 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:27:51.410586 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 651/1318] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:27:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:27:51.415909 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9923e8-a63c-4ccc-97f1-3f571117e0ac None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:27:51.416363 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9923e8-a63c-4ccc-97f1-3f571117e0ac None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:27:51.416461 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9923e8-a63c-4ccc-97f1-3f571117e0ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:27:51.485524 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-2e9923e8-a63c-4ccc-97f1-3f571117e0ac None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:27:51.486756 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-2e9923e8-a63c-4ccc-97f1-3f571117e0ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:27:51.588536 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 668/1319] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:27:51 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:27:51.647543 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c96ffff-c95f-457a-a794-6e648012d0d2 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:27:51.647868 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c96ffff-c95f-457a-a794-6e648012d0d2 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:27:51.648172 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c96ffff-c95f-457a-a794-6e648012d0d2 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:27:51.649846 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 652/1320] 10.222.0.99 () {54 vars in 964 bytes} [Thu Jul 14 15:27:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:27:51.658595 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87874291-3971-459d-9f92-46f963178877 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:27:51.658937 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87874291-3971-459d-9f92-46f963178877 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:27:51.659251 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87874291-3971-459d-9f92-46f963178877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:27:51.681078 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:27:51.744841 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-87874291-3971-459d-9f92-46f963178877 None None] MFA Rules not processed for user `0d0261759f9743b1a13a760ca9159305`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:27:51.746060 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-87874291-3971-459d-9f92-46f963178877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:27:51.822310 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 669/1321] 10.222.0.99 () {58 vars in 1061 bytes} [Thu Jul 14 15:27:51 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:27:51.831438 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c96ffff-c95f-457a-a794-6e648012d0d2 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:27:51.833053 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-5c96ffff-c95f-457a-a794-6e648012d0d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:27:51.941109 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c96ffff-c95f-457a-a794-6e648012d0d2 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:27:52.021509 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e195cf-587c-47b2-beb0-a5494bc17319 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:27:52.022160 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e195cf-587c-47b2-beb0-a5494bc17319 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:27:52.022508 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e195cf-587c-47b2-beb0-a5494bc17319 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:27:52.022755 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e195cf-587c-47b2-beb0-a5494bc17319 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:27:52.024508 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-32e195cf-587c-47b2-beb0-a5494bc17319 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:27:52.033706 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:27:52.151231 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e195cf-587c-47b2-beb0-a5494bc17319 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:27:52.152917 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e195cf-587c-47b2-beb0-a5494bc17319 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:27:52.173738 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 653/1322] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:27:51 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:27:55.489509 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb22dcce-2c9c-4894-9759-9516e2cf2ee2 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:27:55.490045 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb22dcce-2c9c-4894-9759-9516e2cf2ee2 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:27:55.490207 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb22dcce-2c9c-4894-9759-9516e2cf2ee2 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:27:55.491639 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 670/1323] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:27:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:27:55.497094 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64faf0a4-3f52-4a3d-a3d9-cf6bf1210778 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:27:55.497546 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64faf0a4-3f52-4a3d-a3d9-cf6bf1210778 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:27:55.497683 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64faf0a4-3f52-4a3d-a3d9-cf6bf1210778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:27:55.565241 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-64faf0a4-3f52-4a3d-a3d9-cf6bf1210778 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:27:55.566485 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-64faf0a4-3f52-4a3d-a3d9-cf6bf1210778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:27:55.635680 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:27:55.685318 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 654/1324] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:27:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:27:55.725664 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb22dcce-2c9c-4894-9759-9516e2cf2ee2 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:27:55.727404 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-fb22dcce-2c9c-4894-9759-9516e2cf2ee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:27:55.846309 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb22dcce-2c9c-4894-9759-9516e2cf2ee2 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:27:55.860899 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:27:55.925015 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd45218e-f5fb-40bd-b649-1b8b64d2024c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:27:55.925674 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd45218e-f5fb-40bd-b649-1b8b64d2024c service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:27:55.925956 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd45218e-f5fb-40bd-b649-1b8b64d2024c service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:27:55.926237 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd45218e-f5fb-40bd-b649-1b8b64d2024c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:27:55.927915 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-dd45218e-f5fb-40bd-b649-1b8b64d2024c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:27:56.047079 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd45218e-f5fb-40bd-b649-1b8b64d2024c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:27:56.056074 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd45218e-f5fb-40bd-b649-1b8b64d2024c service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:27:56.075676 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 671/1325] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:27:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:27:58.226500 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3657d71-bb90-47e3-86c4-731f6dd51f72 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:27:58.227040 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3657d71-bb90-47e3-86c4-731f6dd51f72 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:27:58.227040 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3657d71-bb90-47e3-86c4-731f6dd51f72 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:27:58.228382 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 655/1326] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:27:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:27:58.233809 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6385d060-dc32-4276-a3ff-02d6f74bdfcb None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:27:58.234253 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6385d060-dc32-4276-a3ff-02d6f74bdfcb None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:27:58.234376 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6385d060-dc32-4276-a3ff-02d6f74bdfcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:27:58.287853 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:27:58.307821 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-6385d060-dc32-4276-a3ff-02d6f74bdfcb None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:27:58.309040 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-6385d060-dc32-4276-a3ff-02d6f74bdfcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:27:58.419358 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 672/1327] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:27:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 15:27:58.456364 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3657d71-bb90-47e3-86c4-731f6dd51f72 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:27:58.458334 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d3657d71-bb90-47e3-86c4-731f6dd51f72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:27:58.569721 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3657d71-bb90-47e3-86c4-731f6dd51f72 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:27:58.651505 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e8f75c-cdef-4b02-b300-450d160711e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:27:58.652192 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e8f75c-cdef-4b02-b300-450d160711e8 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:27:58.652491 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e8f75c-cdef-4b02-b300-450d160711e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:27:58.652776 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e8f75c-cdef-4b02-b300-450d160711e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:27:58.654447 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b3e8f75c-cdef-4b02-b300-450d160711e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:27:58.674022 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:27:58.792471 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e8f75c-cdef-4b02-b300-450d160711e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:27:58.793939 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e8f75c-cdef-4b02-b300-450d160711e8 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:27:58.815194 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 656/1328] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:27:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:28:00.983310 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68b8700-7448-433c-8bc4-a010065cfc8a None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:28:00.984575 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68b8700-7448-433c-8bc4-a010065cfc8a None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:28:00.984738 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68b8700-7448-433c-8bc4-a010065cfc8a None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:28:00.986196 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 673/1329] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:28:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:28:00.992182 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c61b78-2c6c-404f-bb02-eee09acb7ea2 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:28:00.992638 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c61b78-2c6c-404f-bb02-eee09acb7ea2 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:28:00.992786 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c61b78-2c6c-404f-bb02-eee09acb7ea2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:28:01.068658 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-b3c61b78-2c6c-404f-bb02-eee09acb7ea2 None None] MFA Rules not processed for user `7f5177fcb1524b918c6662c934105643`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:28:01.069896 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b3c61b78-2c6c-404f-bb02-eee09acb7ea2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:28:01.144572 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:28:01.185727 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 657/1330] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:28:00 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:28:01.224799 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68b8700-7448-433c-8bc4-a010065cfc8a None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:28:01.226349 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-e68b8700-7448-433c-8bc4-a010065cfc8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:28:01.344153 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68b8700-7448-433c-8bc4-a010065cfc8a None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:28:01.365104 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:28:01.433885 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1918c196-0e0e-4010-a804-fdf9bc5081d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:28:01.433885 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1918c196-0e0e-4010-a804-fdf9bc5081d1 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:28:01.433885 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1918c196-0e0e-4010-a804-fdf9bc5081d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:28:01.433885 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1918c196-0e0e-4010-a804-fdf9bc5081d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:28:01.434477 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-1918c196-0e0e-4010-a804-fdf9bc5081d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:28:01.565676 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1918c196-0e0e-4010-a804-fdf9bc5081d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:28:01.567147 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1918c196-0e0e-4010-a804-fdf9bc5081d1 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:28:01.585950 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 674/1331] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:28:01 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:28:03.972211 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8dd0cd-dcce-4d90-b711-981b496370c0 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:28:03.972844 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8dd0cd-dcce-4d90-b711-981b496370c0 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:28:03.972952 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8dd0cd-dcce-4d90-b711-981b496370c0 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:28:03.974526 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 658/1332] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:28:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:28:03.980266 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501034e5-391f-4482-b88e-45a3480fa1d3 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:28:03.980711 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501034e5-391f-4482-b88e-45a3480fa1d3 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:28:03.980905 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501034e5-391f-4482-b88e-45a3480fa1d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:28:04.050436 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:28:04.053765 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-501034e5-391f-4482-b88e-45a3480fa1d3 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:28:04.055396 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-501034e5-391f-4482-b88e-45a3480fa1d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:28:04.154926 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 675/1333] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:28:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:28:04.191974 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8dd0cd-dcce-4d90-b711-981b496370c0 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:28:04.193532 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-2e8dd0cd-dcce-4d90-b711-981b496370c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:28:04.320261 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8dd0cd-dcce-4d90-b711-981b496370c0 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:28:04.409068 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cef3838-c24c-41a5-9aa2-c37b49fa2f63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:28:04.409716 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cef3838-c24c-41a5-9aa2-c37b49fa2f63 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:28:04.410047 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cef3838-c24c-41a5-9aa2-c37b49fa2f63 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:28:04.410275 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cef3838-c24c-41a5-9aa2-c37b49fa2f63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:28:04.411967 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0cef3838-c24c-41a5-9aa2-c37b49fa2f63 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:28:04.441798 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:28:04.528835 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cef3838-c24c-41a5-9aa2-c37b49fa2f63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:28:04.530313 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cef3838-c24c-41a5-9aa2-c37b49fa2f63 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:28:04.548318 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 659/1334] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:28:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:28:06.684211 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31846352-27ad-43c6-9b6e-4c4afb71d0f3 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:28:06.684745 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31846352-27ad-43c6-9b6e-4c4afb71d0f3 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:28:06.684867 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31846352-27ad-43c6-9b6e-4c4afb71d0f3 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:28:06.686147 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 676/1335] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:28:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:28:06.691885 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81edadc7-6622-49fa-b8c3-72f99d2b59c9 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:28:06.692330 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81edadc7-6622-49fa-b8c3-72f99d2b59c9 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:28:06.692461 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81edadc7-6622-49fa-b8c3-72f99d2b59c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:28:06.756412 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-81edadc7-6622-49fa-b8c3-72f99d2b59c9 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:28:06.757629 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-81edadc7-6622-49fa-b8c3-72f99d2b59c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:28:06.832137 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:28:06.855340 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 660/1336] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:28:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:28:06.897047 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31846352-27ad-43c6-9b6e-4c4afb71d0f3 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:28:06.898624 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-31846352-27ad-43c6-9b6e-4c4afb71d0f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:28:07.004344 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31846352-27ad-43c6-9b6e-4c4afb71d0f3 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:28:07.033380 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:28:07.079998 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6104d0c0-192c-4e0a-b288-dd8f90e7d882 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:28:07.080650 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6104d0c0-192c-4e0a-b288-dd8f90e7d882 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:28:07.080938 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6104d0c0-192c-4e0a-b288-dd8f90e7d882 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:28:07.081218 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6104d0c0-192c-4e0a-b288-dd8f90e7d882 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:28:07.082865 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-6104d0c0-192c-4e0a-b288-dd8f90e7d882 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:28:07.201315 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6104d0c0-192c-4e0a-b288-dd8f90e7d882 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:28:07.202836 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6104d0c0-192c-4e0a-b288-dd8f90e7d882 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:28:07.225981 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 677/1337] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:28:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:28:09.631328 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4dca5b-8d8a-45d3-a294-1a41e1268366 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:28:09.631898 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4dca5b-8d8a-45d3-a294-1a41e1268366 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:28:09.631898 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4dca5b-8d8a-45d3-a294-1a41e1268366 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:28:09.633289 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 661/1338] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:28:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:28:09.638685 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aab930c-8cd3-4127-969c-ddfc231cf00c None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:28:09.639115 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aab930c-8cd3-4127-969c-ddfc231cf00c None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:28:09.639259 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aab930c-8cd3-4127-969c-ddfc231cf00c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:28:09.703996 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-5aab930c-8cd3-4127-969c-ddfc231cf00c None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:28:09.705233 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-5aab930c-8cd3-4127-969c-ddfc231cf00c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:28:09.715393 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:28:09.816073 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 678/1339] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:28:09 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:28:09.853739 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4dca5b-8d8a-45d3-a294-1a41e1268366 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:28:09.855338 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ff4dca5b-8d8a-45d3-a294-1a41e1268366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:28:09.964310 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4dca5b-8d8a-45d3-a294-1a41e1268366 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:28:10.043869 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba54bbb-9a6e-4ae7-8cd8-94cb3f1de6d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:28:10.044520 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba54bbb-9a6e-4ae7-8cd8-94cb3f1de6d8 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:28:10.044808 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba54bbb-9a6e-4ae7-8cd8-94cb3f1de6d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:28:10.045088 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba54bbb-9a6e-4ae7-8cd8-94cb3f1de6d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:28:10.047402 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-aba54bbb-9a6e-4ae7-8cd8-94cb3f1de6d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:28:10.087139 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:28:10.165450 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba54bbb-9a6e-4ae7-8cd8-94cb3f1de6d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:28:10.166887 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba54bbb-9a6e-4ae7-8cd8-94cb3f1de6d8 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:28:10.185770 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 662/1340] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:28:09 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:28:14.420174 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacaba33-d61f-4645-ac91-15b106ac1c9a None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:28:14.420901 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacaba33-d61f-4645-ac91-15b106ac1c9a None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:28:14.420901 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacaba33-d61f-4645-ac91-15b106ac1c9a None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:28:14.423352 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 679/1341] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:28:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:28:14.429083 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40114657-dae0-4010-aa6c-6869d4e52466 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:28:14.429571 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40114657-dae0-4010-aa6c-6869d4e52466 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:28:14.429653 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40114657-dae0-4010-aa6c-6869d4e52466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:28:14.505736 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-40114657-dae0-4010-aa6c-6869d4e52466 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:28:14.507718 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-40114657-dae0-4010-aa6c-6869d4e52466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:28:14.608117 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:28:14.620789 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 663/1342] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:28:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:28:14.663585 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacaba33-d61f-4645-ac91-15b106ac1c9a None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:28:14.665207 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-bacaba33-d61f-4645-ac91-15b106ac1c9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:28:14.781194 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacaba33-d61f-4645-ac91-15b106ac1c9a None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:28:14.819158 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:28:14.858002 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7df313-4c04-42d4-933f-82b486fe98d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:28:14.858738 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7df313-4c04-42d4-933f-82b486fe98d0 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:28:14.859076 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7df313-4c04-42d4-933f-82b486fe98d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:28:14.859358 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7df313-4c04-42d4-933f-82b486fe98d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:28:14.861108 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-ea7df313-4c04-42d4-933f-82b486fe98d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:28:14.977497 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7df313-4c04-42d4-933f-82b486fe98d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:28:14.978955 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7df313-4c04-42d4-933f-82b486fe98d0 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:28:15.003619 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 680/1343] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:28:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:28:17.949157 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722db678-f459-43ce-a5b2-5efbd6e63d9e None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:28:17.949850 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722db678-f459-43ce-a5b2-5efbd6e63d9e None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:28:17.949850 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722db678-f459-43ce-a5b2-5efbd6e63d9e None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:28:17.951144 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 664/1344] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:28:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:28:17.957386 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9f7169-f07f-4e3a-87ad-65c0951846fd None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:28:17.957949 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9f7169-f07f-4e3a-87ad-65c0951846fd None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:28:17.957949 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9f7169-f07f-4e3a-87ad-65c0951846fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:28:18.034878 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-2e9f7169-f07f-4e3a-87ad-65c0951846fd None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:28:18.036138 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-2e9f7169-f07f-4e3a-87ad-65c0951846fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:28:18.055354 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:28:18.145855 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 681/1345] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:28:17 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:28:18.183345 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722db678-f459-43ce-a5b2-5efbd6e63d9e None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:28:18.184934 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-722db678-f459-43ce-a5b2-5efbd6e63d9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:28:18.291906 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722db678-f459-43ce-a5b2-5efbd6e63d9e None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:28:18.378218 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0786a44b-c493-4bd8-a96f-5942a5ed7a19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:28:18.379520 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0786a44b-c493-4bd8-a96f-5942a5ed7a19 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:28:18.379841 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0786a44b-c493-4bd8-a96f-5942a5ed7a19 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:28:18.380709 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0786a44b-c493-4bd8-a96f-5942a5ed7a19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:28:18.383072 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0786a44b-c493-4bd8-a96f-5942a5ed7a19 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:28:18.429584 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:28:18.499907 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0786a44b-c493-4bd8-a96f-5942a5ed7a19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:28:18.501289 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0786a44b-c493-4bd8-a96f-5942a5ed7a19 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:28:18.520655 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 665/1346] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:28:18 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:28:22.903759 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06557545-6032-4f51-9a02-3cc57e5496a9 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:28:22.904380 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06557545-6032-4f51-9a02-3cc57e5496a9 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:28:22.904380 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06557545-6032-4f51-9a02-3cc57e5496a9 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:28:22.906466 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 682/1347] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:28:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:28:22.911892 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a04a09-334a-45e0-9abd-7891fce6bf79 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:28:22.912580 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a04a09-334a-45e0-9abd-7891fce6bf79 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:28:22.912580 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a04a09-334a-45e0-9abd-7891fce6bf79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:28:22.992927 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-a1a04a09-334a-45e0-9abd-7891fce6bf79 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:28:22.995079 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a1a04a09-334a-45e0-9abd-7891fce6bf79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:28:23.121966 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:28:23.131799 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 666/1348] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:28:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 15:28:23.177239 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06557545-6032-4f51-9a02-3cc57e5496a9 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:28:23.179463 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-06557545-6032-4f51-9a02-3cc57e5496a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:28:23.300390 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06557545-6032-4f51-9a02-3cc57e5496a9 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:28:23.356248 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:28:23.384763 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e1a256-5384-419e-8220-600bef7f60e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:28:23.385574 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e1a256-5384-419e-8220-600bef7f60e1 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:28:23.385898 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e1a256-5384-419e-8220-600bef7f60e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:28:23.386174 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e1a256-5384-419e-8220-600bef7f60e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:28:23.387939 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-65e1a256-5384-419e-8220-600bef7f60e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:28:23.508349 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e1a256-5384-419e-8220-600bef7f60e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:28:23.509824 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e1a256-5384-419e-8220-600bef7f60e1 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:28:23.530949 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 683/1349] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:28:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:28:28.613555 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843cb52d-0ff3-4eca-90a1-5276e57fe79c None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:28:28.615162 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843cb52d-0ff3-4eca-90a1-5276e57fe79c None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:28:28.615633 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843cb52d-0ff3-4eca-90a1-5276e57fe79c None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:28:28.618048 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 667/1350] 10.222.0.99 () {54 vars in 921 bytes} [Thu Jul 14 15:28:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:28:28.624363 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f18814-c491-428d-9334-ec6ba4b9a3b8 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:28:28.625259 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f18814-c491-428d-9334-ec6ba4b9a3b8 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:28:28.625786 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f18814-c491-428d-9334-ec6ba4b9a3b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:28:28.718810 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-52f18814-c491-428d-9334-ec6ba4b9a3b8 None None] MFA Rules not processed for user `dc157c485f794e968a60d8405c9310ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:28:28.721714 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-52f18814-c491-428d-9334-ec6ba4b9a3b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:28:28.786470 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:28:28.899468 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 684/1351] 10.222.0.99 () {58 vars in 1018 bytes} [Thu Jul 14 15:28:28 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:28:33.324758 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0462bc-9181-4508-b358-1b0f94b10f8e None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:28:33.325634 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0462bc-9181-4508-b358-1b0f94b10f8e None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:28:33.325916 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0462bc-9181-4508-b358-1b0f94b10f8e None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:28:33.328044 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 668/1352] 10.222.0.99 () {54 vars in 921 bytes} [Thu Jul 14 15:28:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:28:33.334869 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03787d7e-b391-41c4-8a4d-f5379ce9691a None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:28:33.335375 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03787d7e-b391-41c4-8a4d-f5379ce9691a None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:28:33.336251 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03787d7e-b391-41c4-8a4d-f5379ce9691a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:28:33.435907 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-03787d7e-b391-41c4-8a4d-f5379ce9691a None None] MFA Rules not processed for user `dc157c485f794e968a60d8405c9310ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:28:33.437193 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-03787d7e-b391-41c4-8a4d-f5379ce9691a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:28:33.535107 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:28:33.592093 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 685/1353] 10.222.0.99 () {58 vars in 1018 bytes} [Thu Jul 14 15:28:33 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:28:35.072248 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9474872b-f0ca-4770-8ad9-0128cbcfa5fd None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:28:35.072893 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9474872b-f0ca-4770-8ad9-0128cbcfa5fd None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:28:35.072893 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9474872b-f0ca-4770-8ad9-0128cbcfa5fd None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:28:35.074999 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 669/1354] 10.222.0.99 () {54 vars in 921 bytes} [Thu Jul 14 15:28:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:28:35.081406 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa990c1-df85-4379-adde-1a9824a29c8e None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:28:35.081941 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa990c1-df85-4379-adde-1a9824a29c8e None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:28:35.082023 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa990c1-df85-4379-adde-1a9824a29c8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:28:35.169123 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-1aa990c1-df85-4379-adde-1a9824a29c8e None None] MFA Rules not processed for user `dc157c485f794e968a60d8405c9310ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:28:35.170321 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-1aa990c1-df85-4379-adde-1a9824a29c8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:28:35.288942 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 686/1355] 10.222.0.99 () {58 vars in 1018 bytes} [Thu Jul 14 15:28:35 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:28:39.253792 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8351b39e-d9d4-4974-8f0d-9a52e504e17f None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:28:39.254618 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8351b39e-d9d4-4974-8f0d-9a52e504e17f None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:28:39.255356 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8351b39e-d9d4-4974-8f0d-9a52e504e17f None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:28:39.259167 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 670/1356] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:28:39 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:28:39.266899 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9acd9d0-93c5-4f8b-bdd4-c88fc109083f None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:28:39.269731 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9acd9d0-93c5-4f8b-bdd4-c88fc109083f None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:28:39.270491 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9acd9d0-93c5-4f8b-bdd4-c88fc109083f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:28:39.353073 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:28:39.424554 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-a9acd9d0-93c5-4f8b-bdd4-c88fc109083f None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:28:39.426362 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-a9acd9d0-93c5-4f8b-bdd4-c88fc109083f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:28:39.604901 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 687/1357] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:28:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 341 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:28:39.687164 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90771b3-1d67-4c5c-9e8e-7b7b85b3f74e None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:28:39.688327 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90771b3-1d67-4c5c-9e8e-7b7b85b3f74e None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:28:39.688919 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90771b3-1d67-4c5c-9e8e-7b7b85b3f74e None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:28:39.692333 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 671/1358] 10.222.0.99 () {54 vars in 961 bytes} [Thu Jul 14 15:28:39 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:28:39.701149 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714b3277-ce19-404e-81c2-da6e8d793309 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:28:39.701890 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714b3277-ce19-404e-81c2-da6e8d793309 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:28:39.702519 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714b3277-ce19-404e-81c2-da6e8d793309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:28:39.813208 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-714b3277-ce19-404e-81c2-da6e8d793309 None None] MFA Rules not processed for user `3a9b1c2622934c5185f3b5f1ea623bc0`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:28:39.815305 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-714b3277-ce19-404e-81c2-da6e8d793309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:28:39.883023 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:28:39.976628 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 688/1359] 10.222.0.99 () {58 vars in 1058 bytes} [Thu Jul 14 15:28:39 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:28:39.986598 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90771b3-1d67-4c5c-9e8e-7b7b85b3f74e None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:28:39.987103 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f90771b3-1d67-4c5c-9e8e-7b7b85b3f74e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:28:40.233794 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90771b3-1d67-4c5c-9e8e-7b7b85b3f74e None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:28:40.318632 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:28:40.367584 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8487a36a-b1d7-49d7-b70d-0c737466cec8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9b1c2622934c5185f3b5f1ea623bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:28:40.368412 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8487a36a-b1d7-49d7-b70d-0c737466cec8 service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:28:40.368810 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8487a36a-b1d7-49d7-b70d-0c737466cec8 service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:28:40.369171 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8487a36a-b1d7-49d7-b70d-0c737466cec8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:28:40.371056 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-8487a36a-b1d7-49d7-b70d-0c737466cec8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:28:40.524761 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8487a36a-b1d7-49d7-b70d-0c737466cec8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:28:40.527611 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8487a36a-b1d7-49d7-b70d-0c737466cec8 service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:28:40.553907 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 672/1360] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:28:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 569 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:28:42.997366 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdd1501-7785-4d3b-b4a8-f26c9865b7c1 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:28:42.997814 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdd1501-7785-4d3b-b4a8-f26c9865b7c1 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:28:42.998056 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdd1501-7785-4d3b-b4a8-f26c9865b7c1 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:28:43.000382 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 689/1361] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:28:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:28:43.006332 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb7f9cb-b8de-4653-8b47-df664db20f8b None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:28:43.006783 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb7f9cb-b8de-4653-8b47-df664db20f8b None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:28:43.007065 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb7f9cb-b8de-4653-8b47-df664db20f8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:28:43.090281 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-3bb7f9cb-b8de-4653-8b47-df664db20f8b None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:28:43.091522 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-3bb7f9cb-b8de-4653-8b47-df664db20f8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:28:43.146196 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:28:43.271997 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 673/1362] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:28:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:28:43.323490 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdd1501-7785-4d3b-b4a8-f26c9865b7c1 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:28:43.323490 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-bfdd1501-7785-4d3b-b4a8-f26c9865b7c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:28:43.546271 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdd1501-7785-4d3b-b4a8-f26c9865b7c1 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:28:43.558414 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:28:43.690053 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84862003-d518-4f1d-88e8-19153da86187 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9b1c2622934c5185f3b5f1ea623bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:28:43.690953 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84862003-d518-4f1d-88e8-19153da86187 service nova] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:28:43.691225 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84862003-d518-4f1d-88e8-19153da86187 service nova] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:28:43.691704 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84862003-d518-4f1d-88e8-19153da86187 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:28:43.695947 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-84862003-d518-4f1d-88e8-19153da86187 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:28:43.816341 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84862003-d518-4f1d-88e8-19153da86187 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:28:43.818479 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84862003-d518-4f1d-88e8-19153da86187 service nova] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:28:43.838195 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 690/1363] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:28:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 518 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:28:46.324867 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac61bd1-ad67-4fce-981c-cd262bc1c3b2 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:28:46.325423 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac61bd1-ad67-4fce-981c-cd262bc1c3b2 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:28:46.325423 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac61bd1-ad67-4fce-981c-cd262bc1c3b2 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:28:46.327103 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 674/1364] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:28:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:28:46.333532 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce29d3bd-df65-464c-bfd4-11ef67021baf None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:28:46.333532 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce29d3bd-df65-464c-bfd4-11ef67021baf None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:28:46.333532 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce29d3bd-df65-464c-bfd4-11ef67021baf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:28:46.349174 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:28:46.419896 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-ce29d3bd-df65-464c-bfd4-11ef67021baf None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:28:46.422469 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-ce29d3bd-df65-464c-bfd4-11ef67021baf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:28:46.557748 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 691/1365] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:28:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:28:46.611145 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b2dc71-1caa-4ff0-80bf-e7ea29b7ae07 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:28:46.611823 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b2dc71-1caa-4ff0-80bf-e7ea29b7ae07 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:28:46.612317 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b2dc71-1caa-4ff0-80bf-e7ea29b7ae07 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:28:46.613949 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 675/1366] 10.222.0.99 () {54 vars in 961 bytes} [Thu Jul 14 15:28:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 15:28:46.620251 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d259807f-f3a5-49d5-87c0-1e0ec5069f05 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:28:46.620545 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d259807f-f3a5-49d5-87c0-1e0ec5069f05 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:28:46.620821 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d259807f-f3a5-49d5-87c0-1e0ec5069f05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:28:46.729227 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-d259807f-f3a5-49d5-87c0-1e0ec5069f05 None None] MFA Rules not processed for user `3a9b1c2622934c5185f3b5f1ea623bc0`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:28:46.730857 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-d259807f-f3a5-49d5-87c0-1e0ec5069f05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:28:46.782125 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:28:46.905495 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 692/1367] 10.222.0.99 () {58 vars in 1058 bytes} [Thu Jul 14 15:28:46 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 15:28:46.913653 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b2dc71-1caa-4ff0-80bf-e7ea29b7ae07 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:28:46.915778 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-05b2dc71-1caa-4ff0-80bf-e7ea29b7ae07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:28:47.095526 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b2dc71-1caa-4ff0-80bf-e7ea29b7ae07 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:28:47.117830 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:28:47.245993 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7038e36d-d296-4313-a39f-b6019f158ad8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9b1c2622934c5185f3b5f1ea623bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:28:47.246658 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7038e36d-d296-4313-a39f-b6019f158ad8 service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:28:47.246946 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7038e36d-d296-4313-a39f-b6019f158ad8 service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:28:47.247223 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7038e36d-d296-4313-a39f-b6019f158ad8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:28:47.250337 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-7038e36d-d296-4313-a39f-b6019f158ad8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:28:47.363903 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7038e36d-d296-4313-a39f-b6019f158ad8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:28:47.365564 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7038e36d-d296-4313-a39f-b6019f158ad8 service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:28:47.388438 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 676/1368] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:28:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 15:28:48.329083 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b977cda-175d-4a43-a2b5-6bd92028b478 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:28:48.329612 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b977cda-175d-4a43-a2b5-6bd92028b478 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:28:48.329895 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b977cda-175d-4a43-a2b5-6bd92028b478 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:28:48.331725 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 693/1369] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:28:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:28:48.338378 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a512f0bd-81ca-4736-bacc-71d12a914d4a None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:28:48.338749 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a512f0bd-81ca-4736-bacc-71d12a914d4a None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:28:48.339076 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a512f0bd-81ca-4736-bacc-71d12a914d4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:28:48.379512 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:28:48.429519 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-a512f0bd-81ca-4736-bacc-71d12a914d4a None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:28:48.431541 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a512f0bd-81ca-4736-bacc-71d12a914d4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:28:48.554936 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 677/1370] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:28:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:28:48.915888 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec641981-1678-4d54-8818-4ba7e887988f None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:28:48.915888 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec641981-1678-4d54-8818-4ba7e887988f None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:28:48.915888 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec641981-1678-4d54-8818-4ba7e887988f None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:28:48.918717 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 694/1371] 10.222.0.99 () {54 vars in 963 bytes} [Thu Jul 14 15:28:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:28:48.924067 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a964a6ff-9321-4077-9172-3e8b923f400b None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:28:48.924067 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a964a6ff-9321-4077-9172-3e8b923f400b None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:28:48.924067 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a964a6ff-9321-4077-9172-3e8b923f400b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:28:49.013268 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-a964a6ff-9321-4077-9172-3e8b923f400b None None] MFA Rules not processed for user `a3e1a2b83f3d4d88ba5084dfaffd22f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:28:49.015106 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a964a6ff-9321-4077-9172-3e8b923f400b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:28:49.073961 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:28:49.113282 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 678/1372] 10.222.0.99 () {58 vars in 1060 bytes} [Thu Jul 14 15:28:48 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:28:49.122920 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec641981-1678-4d54-8818-4ba7e887988f None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:28:49.125198 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-ec641981-1678-4d54-8818-4ba7e887988f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:28:49.268798 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec641981-1678-4d54-8818-4ba7e887988f None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:28:49.322542 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:28:49.362344 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a94faf8-b9ec-480a-b686-9bdbda63e6a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3e1a2b83f3d4d88ba5084dfaffd22f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:28:49.363224 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a94faf8-b9ec-480a-b686-9bdbda63e6a2 service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:28:49.363323 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a94faf8-b9ec-480a-b686-9bdbda63e6a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:28:49.363615 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a94faf8-b9ec-480a-b686-9bdbda63e6a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:28:49.365403 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-1a94faf8-b9ec-480a-b686-9bdbda63e6a2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:28:49.489620 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a94faf8-b9ec-480a-b686-9bdbda63e6a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:28:49.491082 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a94faf8-b9ec-480a-b686-9bdbda63e6a2 service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:28:49.512307 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 695/1373] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:28:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:28:49.705211 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debb248b-a753-4327-89d4-16501fd5a385 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:28:49.705737 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debb248b-a753-4327-89d4-16501fd5a385 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:28:49.705981 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debb248b-a753-4327-89d4-16501fd5a385 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:28:49.707260 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 679/1374] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:28:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:28:49.713349 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a151fc7b-057b-4d4b-ac10-bdad8c1024a3 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:28:49.714129 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a151fc7b-057b-4d4b-ac10-bdad8c1024a3 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:28:49.714376 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a151fc7b-057b-4d4b-ac10-bdad8c1024a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:28:49.817030 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-a151fc7b-057b-4d4b-ac10-bdad8c1024a3 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:28:49.818749 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-a151fc7b-057b-4d4b-ac10-bdad8c1024a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:28:49.844809 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:28:49.965295 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 696/1375] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:28:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:28:50.018486 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debb248b-a753-4327-89d4-16501fd5a385 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:28:50.019960 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-debb248b-a753-4327-89d4-16501fd5a385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:28:50.209527 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debb248b-a753-4327-89d4-16501fd5a385 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:28:50.279243 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:28:50.388141 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec73dcd-4076-440e-a77d-19594d911a6a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9b1c2622934c5185f3b5f1ea623bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:28:50.388804 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec73dcd-4076-440e-a77d-19594d911a6a service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:28:50.389130 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec73dcd-4076-440e-a77d-19594d911a6a service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:28:50.389564 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec73dcd-4076-440e-a77d-19594d911a6a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:28:50.391962 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-aec73dcd-4076-440e-a77d-19594d911a6a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:28:50.507264 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec73dcd-4076-440e-a77d-19594d911a6a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:28:50.509946 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec73dcd-4076-440e-a77d-19594d911a6a service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:28:50.532968 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 680/1376] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:28:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 515 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:28:52.695638 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5872fc4d-6433-4630-811b-4d75c6b8623a None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:28:52.696153 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5872fc4d-6433-4630-811b-4d75c6b8623a None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:28:52.696643 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5872fc4d-6433-4630-811b-4d75c6b8623a None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:28:52.699135 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 697/1377] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:28:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:28:52.705236 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b6673b-fbcf-4f59-849d-9ae6be2183f7 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:28:52.705726 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b6673b-fbcf-4f59-849d-9ae6be2183f7 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:28:52.705851 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b6673b-fbcf-4f59-849d-9ae6be2183f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:28:52.787945 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-39b6673b-fbcf-4f59-849d-9ae6be2183f7 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:28:52.789592 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-39b6673b-fbcf-4f59-849d-9ae6be2183f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:28:52.798600 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:28:52.900933 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 681/1378] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:28:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:28:52.945816 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5872fc4d-6433-4630-811b-4d75c6b8623a None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:28:52.947254 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-5872fc4d-6433-4630-811b-4d75c6b8623a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:28:53.127301 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5872fc4d-6433-4630-811b-4d75c6b8623a None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:28:53.135755 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:28:53.308151 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b0faaa-70b1-4644-92fb-3188847a2c7e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9b1c2622934c5185f3b5f1ea623bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:28:53.309104 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b0faaa-70b1-4644-92fb-3188847a2c7e service nova] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:28:53.309552 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b0faaa-70b1-4644-92fb-3188847a2c7e service nova] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:28:53.309983 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b0faaa-70b1-4644-92fb-3188847a2c7e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:28:53.313409 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-20b0faaa-70b1-4644-92fb-3188847a2c7e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:28:53.414307 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b0faaa-70b1-4644-92fb-3188847a2c7e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:28:53.415947 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b0faaa-70b1-4644-92fb-3188847a2c7e service nova] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:28:53.435512 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 698/1379] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:28:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:28:55.575634 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302f62dd-cd22-489a-a009-381633bd5bac None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:28:55.576173 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302f62dd-cd22-489a-a009-381633bd5bac None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:28:55.576173 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302f62dd-cd22-489a-a009-381633bd5bac None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:28:55.578050 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 682/1380] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:28:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 15:28:55.583393 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8fd10d-641d-40df-8b9b-3dbf0a882564 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:28:55.583873 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8fd10d-641d-40df-8b9b-3dbf0a882564 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:28:55.584106 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8fd10d-641d-40df-8b9b-3dbf0a882564 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:28:55.594752 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:28:55.653775 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-fc8fd10d-641d-40df-8b9b-3dbf0a882564 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:28:55.655588 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-fc8fd10d-641d-40df-8b9b-3dbf0a882564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:28:55.767537 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 699/1381] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:28:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:28:55.812780 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302f62dd-cd22-489a-a009-381633bd5bac None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:28:55.814306 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-302f62dd-cd22-489a-a009-381633bd5bac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:28:55.956096 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:28:55.968421 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302f62dd-cd22-489a-a009-381633bd5bac None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:28:56.100030 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863334bf-73f2-414d-8505-95b867a097e5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9b1c2622934c5185f3b5f1ea623bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:28:56.100690 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863334bf-73f2-414d-8505-95b867a097e5 service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:28:56.100978 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863334bf-73f2-414d-8505-95b867a097e5 service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:28:56.101258 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863334bf-73f2-414d-8505-95b867a097e5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:28:56.102900 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-863334bf-73f2-414d-8505-95b867a097e5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:28:56.208907 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863334bf-73f2-414d-8505-95b867a097e5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:28:56.210462 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863334bf-73f2-414d-8505-95b867a097e5 service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:28:56.225209 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:28:56.237797 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 683/1382] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:28:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:28:58.397444 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194e3a39-f4a0-4210-a5d6-f293645f1f2e None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:28:58.397986 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194e3a39-f4a0-4210-a5d6-f293645f1f2e None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:28:58.398083 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194e3a39-f4a0-4210-a5d6-f293645f1f2e None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:28:58.399695 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 700/1383] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:28:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:28:58.405690 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb7a20a-e483-490d-9fdb-85ca1c2c1fdc None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:28:58.407166 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb7a20a-e483-490d-9fdb-85ca1c2c1fdc None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:28:58.407166 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb7a20a-e483-490d-9fdb-85ca1c2c1fdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:28:58.489034 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-1eb7a20a-e483-490d-9fdb-85ca1c2c1fdc None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:28:58.490786 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-1eb7a20a-e483-490d-9fdb-85ca1c2c1fdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:28:58.613233 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 684/1384] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:28:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:28:58.657834 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194e3a39-f4a0-4210-a5d6-f293645f1f2e None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:28:58.659297 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-194e3a39-f4a0-4210-a5d6-f293645f1f2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:28:58.747122 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:28:58.825710 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194e3a39-f4a0-4210-a5d6-f293645f1f2e None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:28:58.958585 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a121634b-77d9-449f-97e6-b42c80bdac53 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9b1c2622934c5185f3b5f1ea623bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:28:58.959188 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a121634b-77d9-449f-97e6-b42c80bdac53 service nova] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:28:58.959484 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a121634b-77d9-449f-97e6-b42c80bdac53 service nova] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:28:58.959803 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a121634b-77d9-449f-97e6-b42c80bdac53 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:28:58.961538 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-a121634b-77d9-449f-97e6-b42c80bdac53 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:28:59.014891 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:28:59.057225 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a121634b-77d9-449f-97e6-b42c80bdac53 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:28:59.058837 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a121634b-77d9-449f-97e6-b42c80bdac53 service nova] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:28:59.081030 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 701/1385] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:28:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:29:01.282672 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41755479-90c0-42d5-8f87-58a344af6994 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:29:01.283329 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41755479-90c0-42d5-8f87-58a344af6994 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:29:01.283329 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41755479-90c0-42d5-8f87-58a344af6994 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:29:01.284650 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 685/1386] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:29:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:29:01.290116 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68a62dc-a581-434f-ae01-e4dc77742e63 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:29:01.290620 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68a62dc-a581-434f-ae01-e4dc77742e63 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:29:01.290790 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68a62dc-a581-434f-ae01-e4dc77742e63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:29:01.357010 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-c68a62dc-a581-434f-ae01-e4dc77742e63 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:29:01.358892 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-c68a62dc-a581-434f-ae01-e4dc77742e63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:29:01.464544 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 702/1387] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:29:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:29:01.510139 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41755479-90c0-42d5-8f87-58a344af6994 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:29:01.511701 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-41755479-90c0-42d5-8f87-58a344af6994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:29:01.593610 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:29:01.665632 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41755479-90c0-42d5-8f87-58a344af6994 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:29:01.782562 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0954685-0eee-4c28-9424-ccc848c12102 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9b1c2622934c5185f3b5f1ea623bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:29:01.783215 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0954685-0eee-4c28-9424-ccc848c12102 service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:29:01.783504 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0954685-0eee-4c28-9424-ccc848c12102 service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:29:01.783808 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0954685-0eee-4c28-9424-ccc848c12102 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:29:01.785464 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d0954685-0eee-4c28-9424-ccc848c12102 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:29:01.823407 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:29:01.882087 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0954685-0eee-4c28-9424-ccc848c12102 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:29:01.883659 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0954685-0eee-4c28-9424-ccc848c12102 service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:29:01.902943 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 686/1388] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:29:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:29:04.258561 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c17c88a-b17c-48a9-aefc-85cad887d595 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:29:04.259088 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c17c88a-b17c-48a9-aefc-85cad887d595 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:29:04.259088 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c17c88a-b17c-48a9-aefc-85cad887d595 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:29:04.260488 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 703/1389] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:29:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:29:04.269752 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0f4972-de8e-477a-9bbd-a706cf9db436 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:29:04.270538 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0f4972-de8e-477a-9bbd-a706cf9db436 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:29:04.270538 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0f4972-de8e-477a-9bbd-a706cf9db436 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:29:04.339477 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-ce0f4972-de8e-477a-9bbd-a706cf9db436 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:29:04.340750 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ce0f4972-de8e-477a-9bbd-a706cf9db436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:29:04.443580 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 687/1390] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:29:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:29:04.495325 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c17c88a-b17c-48a9-aefc-85cad887d595 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:29:04.497069 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-4c17c88a-b17c-48a9-aefc-85cad887d595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:29:04.528918 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:29:04.649042 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c17c88a-b17c-48a9-aefc-85cad887d595 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:29:04.761765 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:29:04.775616 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4c5e85-3e05-46df-9209-fa4b95a7ca06 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9b1c2622934c5185f3b5f1ea623bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:29:04.776279 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4c5e85-3e05-46df-9209-fa4b95a7ca06 service nova] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:29:04.776560 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4c5e85-3e05-46df-9209-fa4b95a7ca06 service nova] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:29:04.776836 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4c5e85-3e05-46df-9209-fa4b95a7ca06 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:29:04.778496 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-ca4c5e85-3e05-46df-9209-fa4b95a7ca06 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:29:04.872217 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4c5e85-3e05-46df-9209-fa4b95a7ca06 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:29:04.874945 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4c5e85-3e05-46df-9209-fa4b95a7ca06 service nova] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:29:04.894278 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 704/1391] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:29:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:29:05.217915 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3a1347-6cdb-41dd-bca8-91733041f9fa None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:29:05.218190 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3a1347-6cdb-41dd-bca8-91733041f9fa None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:29:05.218481 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3a1347-6cdb-41dd-bca8-91733041f9fa None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:29:05.219975 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 688/1392] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:29:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:29:05.225377 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f666981-1cc3-4701-9986-c9b734562845 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:29:05.225705 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f666981-1cc3-4701-9986-c9b734562845 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:29:05.225990 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f666981-1cc3-4701-9986-c9b734562845 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:29:05.292262 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-4f666981-1cc3-4701-9986-c9b734562845 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:29:05.293502 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-4f666981-1cc3-4701-9986-c9b734562845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:29:05.353198 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:29:05.393998 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 705/1393] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:29:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:29:05.451527 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971425b4-f9df-42e6-b9f7-77fa60c28d23 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:29:05.453185 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971425b4-f9df-42e6-b9f7-77fa60c28d23 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:29:05.453185 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971425b4-f9df-42e6-b9f7-77fa60c28d23 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:29:05.453965 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 689/1394] 10.222.0.99 () {54 vars in 962 bytes} [Thu Jul 14 15:29:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:29:05.459379 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfa1e9d-658e-40b8-be1d-aa1011c49d7e None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:29:05.459697 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfa1e9d-658e-40b8-be1d-aa1011c49d7e None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:29:05.459980 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfa1e9d-658e-40b8-be1d-aa1011c49d7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:29:05.539122 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-8cfa1e9d-658e-40b8-be1d-aa1011c49d7e None None] MFA Rules not processed for user `605e230d50ce44629f554c95b2f79ead`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:29:05.540285 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-8cfa1e9d-658e-40b8-be1d-aa1011c49d7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:29:05.621948 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 706/1395] 10.222.0.99 () {58 vars in 1059 bytes} [Thu Jul 14 15:29:05 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:29:05.630712 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971425b4-f9df-42e6-b9f7-77fa60c28d23 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:29:05.632294 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-971425b4-f9df-42e6-b9f7-77fa60c28d23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:29:05.648026 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:29:05.746174 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971425b4-f9df-42e6-b9f7-77fa60c28d23 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:29:05.822487 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f843d094-b311-4a60-8a56-e135008c1b84 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605e230d50ce44629f554c95b2f79ead', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:29:05.823048 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f843d094-b311-4a60-8a56-e135008c1b84 service glance] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:29:05.823334 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f843d094-b311-4a60-8a56-e135008c1b84 service glance] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:29:05.823625 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f843d094-b311-4a60-8a56-e135008c1b84 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:29:05.825297 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f843d094-b311-4a60-8a56-e135008c1b84 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:29:05.892942 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:29:05.951011 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f843d094-b311-4a60-8a56-e135008c1b84 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:29:05.954170 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f843d094-b311-4a60-8a56-e135008c1b84 service glance] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:29:05.979528 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 690/1396] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:29:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:29:06.714057 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6849063d-536d-4151-8425-4b2cfc071377 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:29:06.714572 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6849063d-536d-4151-8425-4b2cfc071377 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:29:06.714572 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6849063d-536d-4151-8425-4b2cfc071377 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:29:06.783098 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-6849063d-536d-4151-8425-4b2cfc071377 None None] MFA Rules not processed for user `adaeadf613954d7393b3fd3bae68c362`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:29:06.784382 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-6849063d-536d-4151-8425-4b2cfc071377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:29:06.793158 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:29:06.873456 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 707/1397] 10.222.0.99 () {58 vars in 1009 bytes} [Thu Jul 14 15:29:06 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:29:06.922556 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3583f85b-7337-43a3-a3e8-7778805b0783 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:29:06.923047 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3583f85b-7337-43a3-a3e8-7778805b0783 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:29:06.923149 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3583f85b-7337-43a3-a3e8-7778805b0783 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:29:06.924577 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 691/1398] 10.222.0.99 () {54 vars in 942 bytes} [Thu Jul 14 15:29:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:29:06.931058 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7a195a-768e-4413-bdbf-0e94a3ac9d7c None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:29:06.931355 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7a195a-768e-4413-bdbf-0e94a3ac9d7c None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:29:06.931708 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7a195a-768e-4413-bdbf-0e94a3ac9d7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:29:07.018506 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-bc7a195a-768e-4413-bdbf-0e94a3ac9d7c None None] MFA Rules not processed for user `ddd2ec3b3ebc4aacb322c95270374451`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:29:07.019765 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-bc7a195a-768e-4413-bdbf-0e94a3ac9d7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:29:07.036496 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0514514-30b0-478d-8466-6b81bcaed273 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:29:07.036794 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0514514-30b0-478d-8466-6b81bcaed273 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:29:07.037077 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0514514-30b0-478d-8466-6b81bcaed273 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:29:07.038738 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 692/1399] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:29:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 15:29:07.043400 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969b5c2f-47ad-452e-8258-5b4470c49477 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:29:07.043839 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969b5c2f-47ad-452e-8258-5b4470c49477 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:29:07.044253 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969b5c2f-47ad-452e-8258-5b4470c49477 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:29:07.106997 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:29:07.113181 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-969b5c2f-47ad-452e-8258-5b4470c49477 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:29:07.114575 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-969b5c2f-47ad-452e-8258-5b4470c49477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:29:07.130682 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 708/1400] 10.222.0.99 () {58 vars in 1039 bytes} [Thu Jul 14 15:29:06 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:29:07.141609 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7a195a-768e-4413-bdbf-0e94a3ac9d7c None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:29:07.151041 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7a195a-768e-4413-bdbf-0e94a3ac9d7c None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:29:07.215479 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 693/1401] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:29:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 15:29:07.261035 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969b5c2f-47ad-452e-8258-5b4470c49477 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:29:07.262408 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-969b5c2f-47ad-452e-8258-5b4470c49477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:29:07.264921 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83db3bcf-6699-4733-a61a-e5cd98397de7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd2ec3b3ebc4aacb322c95270374451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:29:07.265589 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83db3bcf-6699-4733-a61a-e5cd98397de7 service swift] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:29:07.266004 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83db3bcf-6699-4733-a61a-e5cd98397de7 service swift] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:29:07.266127 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83db3bcf-6699-4733-a61a-e5cd98397de7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:29:07.268119 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-83db3bcf-6699-4733-a61a-e5cd98397de7 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:29:07.291372 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:29:07.356553 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83db3bcf-6699-4733-a61a-e5cd98397de7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:29:07.358882 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83db3bcf-6699-4733-a61a-e5cd98397de7 service swift] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:29:07.367589 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 709/1402] 10.222.0.99 () {60 vars in 1384 bytes} [Thu Jul 14 15:29:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 14 15:29:07.382957 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969b5c2f-47ad-452e-8258-5b4470c49477 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:29:07.541415 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e21ed8-01a6-4ddb-abbe-5f568cefa342 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9b1c2622934c5185f3b5f1ea623bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:29:07.541988 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e21ed8-01a6-4ddb-abbe-5f568cefa342 service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:29:07.542277 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e21ed8-01a6-4ddb-abbe-5f568cefa342 service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:29:07.542561 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e21ed8-01a6-4ddb-abbe-5f568cefa342 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:29:07.544309 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e8e21ed8-01a6-4ddb-abbe-5f568cefa342 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:29:07.569311 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:29:07.651704 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e21ed8-01a6-4ddb-abbe-5f568cefa342 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:29:07.653457 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e21ed8-01a6-4ddb-abbe-5f568cefa342 service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:29:07.673381 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 694/1403] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:29:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 15:29:09.825133 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a15031-a55d-4527-a965-1361dc3103ae None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:29:09.825786 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a15031-a55d-4527-a965-1361dc3103ae None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:29:09.825786 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a15031-a55d-4527-a965-1361dc3103ae None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:29:09.827152 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 710/1404] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:29:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:29:09.832564 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118fad55-4574-4dbc-9712-bad184bd5451 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:29:09.833017 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118fad55-4574-4dbc-9712-bad184bd5451 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:29:09.833148 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118fad55-4574-4dbc-9712-bad184bd5451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:29:09.904277 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-118fad55-4574-4dbc-9712-bad184bd5451 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:29:09.905629 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-118fad55-4574-4dbc-9712-bad184bd5451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:29:10.016875 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:29:10.039838 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 695/1405] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:29:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:29:10.070357 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a15031-a55d-4527-a965-1361dc3103ae None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:29:10.071926 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-e8a15031-a55d-4527-a965-1361dc3103ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:29:10.082830 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:29:10.245307 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a15031-a55d-4527-a965-1361dc3103ae None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:29:10.351089 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:29:10.383711 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72cadf4-5c5b-4fd4-a63e-e5f29ab83ab8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9b1c2622934c5185f3b5f1ea623bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:29:10.384371 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72cadf4-5c5b-4fd4-a63e-e5f29ab83ab8 service nova] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:29:10.384653 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72cadf4-5c5b-4fd4-a63e-e5f29ab83ab8 service nova] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:29:10.384926 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72cadf4-5c5b-4fd4-a63e-e5f29ab83ab8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:29:10.386578 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-c72cadf4-5c5b-4fd4-a63e-e5f29ab83ab8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:29:10.486273 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72cadf4-5c5b-4fd4-a63e-e5f29ab83ab8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:29:10.487919 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72cadf4-5c5b-4fd4-a63e-e5f29ab83ab8 service nova] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:29:10.508427 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 711/1406] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:29:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:29:12.644009 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d275bcf9-4694-4ab2-9313-133fc6c4c1f0 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:29:12.644582 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d275bcf9-4694-4ab2-9313-133fc6c4c1f0 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:29:12.644692 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d275bcf9-4694-4ab2-9313-133fc6c4c1f0 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:29:12.646100 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 696/1407] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:29:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:29:12.651899 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea894fb-994d-4339-a365-3480f9967732 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:29:12.652317 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea894fb-994d-4339-a365-3480f9967732 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:29:12.652455 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea894fb-994d-4339-a365-3480f9967732 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:29:12.726117 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-6ea894fb-994d-4339-a365-3480f9967732 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:29:12.727352 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-6ea894fb-994d-4339-a365-3480f9967732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:29:12.778971 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:29:12.838219 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 712/1408] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:29:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:29:12.886109 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d275bcf9-4694-4ab2-9313-133fc6c4c1f0 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:29:12.888009 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d275bcf9-4694-4ab2-9313-133fc6c4c1f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:29:13.067662 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d275bcf9-4694-4ab2-9313-133fc6c4c1f0 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:29:13.143552 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:29:13.208177 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdaae26-b830-4466-ad88-1bbefebd50cf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9b1c2622934c5185f3b5f1ea623bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:29:13.208853 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdaae26-b830-4466-ad88-1bbefebd50cf service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:29:13.209195 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdaae26-b830-4466-ad88-1bbefebd50cf service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:29:13.209489 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdaae26-b830-4466-ad88-1bbefebd50cf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:29:13.211199 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-fbdaae26-b830-4466-ad88-1bbefebd50cf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:29:13.322546 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdaae26-b830-4466-ad88-1bbefebd50cf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:29:13.324679 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdaae26-b830-4466-ad88-1bbefebd50cf service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:29:13.344321 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 697/1409] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:29:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:29:22.835646 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=23 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:29:23.341611 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=64 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:29:30.138014 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714b99dd-f54b-4a93-a43c-1ba496ef2834 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:29:30.138573 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714b99dd-f54b-4a93-a43c-1ba496ef2834 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:29:30.138573 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714b99dd-f54b-4a93-a43c-1ba496ef2834 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:29:30.140507 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 713/1410] 10.222.0.99 () {54 vars in 918 bytes} [Thu Jul 14 15:29:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:29:30.145950 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84a9d28-0450-4c18-bd11-b40a0affc012 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:29:30.146428 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84a9d28-0450-4c18-bd11-b40a0affc012 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:29:30.146533 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84a9d28-0450-4c18-bd11-b40a0affc012 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:29:30.229061 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-e84a9d28-0450-4c18-bd11-b40a0affc012 None None] MFA Rules not processed for user `dc157c485f794e968a60d8405c9310ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:29:30.230315 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e84a9d28-0450-4c18-bd11-b40a0affc012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:29:30.354325 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 698/1411] 10.222.0.99 () {58 vars in 1015 bytes} [Thu Jul 14 15:29:30 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:29:30.369386 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4552827c-ab2d-4cc8-893f-3b5279258a87 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:29:30.369680 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4552827c-ab2d-4cc8-893f-3b5279258a87 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:29:30.370032 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4552827c-ab2d-4cc8-893f-3b5279258a87 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:29:30.371457 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 714/1412] 10.222.0.99 () {54 vars in 918 bytes} [Thu Jul 14 15:29:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:29:30.378413 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22020ba2-56e8-4f74-a677-33924be7e370 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:29:30.378780 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22020ba2-56e8-4f74-a677-33924be7e370 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:29:30.379069 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22020ba2-56e8-4f74-a677-33924be7e370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:29:30.473002 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:29:30.477581 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-22020ba2-56e8-4f74-a677-33924be7e370 None None] MFA Rules not processed for user `3a9b1c2622934c5185f3b5f1ea623bc0`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:29:30.478844 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-22020ba2-56e8-4f74-a677-33924be7e370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:29:30.603354 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 699/1413] 10.222.0.99 () {58 vars in 1015 bytes} [Thu Jul 14 15:29:30 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:29:33.181447 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304e9c33-0b79-40a5-b031-db9c4aed43cb None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:29:33.181989 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304e9c33-0b79-40a5-b031-db9c4aed43cb None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:29:33.181989 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304e9c33-0b79-40a5-b031-db9c4aed43cb None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:29:33.183329 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 715/1414] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 15:29:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:29:33.190039 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b08fb8-d2ef-4bc0-b305-c8b68f0aacad None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:29:33.190486 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b08fb8-d2ef-4bc0-b305-c8b68f0aacad None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:29:33.190619 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b08fb8-d2ef-4bc0-b305-c8b68f0aacad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:29:33.259313 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-57b08fb8-d2ef-4bc0-b305-c8b68f0aacad None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:29:33.260592 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-57b08fb8-d2ef-4bc0-b305-c8b68f0aacad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:29:33.318726 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:29:33.362252 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 700/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 15:29:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:29:33.412182 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304e9c33-0b79-40a5-b031-db9c4aed43cb None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:29:33.414208 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-304e9c33-0b79-40a5-b031-db9c4aed43cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:29:33.576402 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304e9c33-0b79-40a5-b031-db9c4aed43cb None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:29:33.653815 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:29:33.697478 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7826a274-6505-491e-be03-592981aa8ecc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9b1c2622934c5185f3b5f1ea623bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:29:33.698141 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7826a274-6505-491e-be03-592981aa8ecc service nova] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:29:33.698429 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7826a274-6505-491e-be03-592981aa8ecc service nova] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:29:33.698713 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7826a274-6505-491e-be03-592981aa8ecc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:29:33.700528 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-7826a274-6505-491e-be03-592981aa8ecc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:29:33.797224 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7826a274-6505-491e-be03-592981aa8ecc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:29:33.798879 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7826a274-6505-491e-be03-592981aa8ecc service nova] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:29:33.817614 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 716/1416] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:29:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:29:34.791296 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc57b415-c672-4042-9d42-44e319201993 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:29:34.791940 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc57b415-c672-4042-9d42-44e319201993 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:29:34.791940 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc57b415-c672-4042-9d42-44e319201993 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:29:34.793361 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 701/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 15:29:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:29:34.799256 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2403a0c5-5298-4baf-86a8-b52084eb96c4 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:29:34.799703 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2403a0c5-5298-4baf-86a8-b52084eb96c4 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:29:34.799932 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2403a0c5-5298-4baf-86a8-b52084eb96c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:29:34.916845 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-2403a0c5-5298-4baf-86a8-b52084eb96c4 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:29:34.918865 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-2403a0c5-5298-4baf-86a8-b52084eb96c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:29:34.964535 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:29:35.039710 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 717/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 15:29:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:29:35.191734 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc57b415-c672-4042-9d42-44e319201993 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:29:35.193259 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-bc57b415-c672-4042-9d42-44e319201993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:29:35.350762 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc57b415-c672-4042-9d42-44e319201993 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:29:35.396606 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:29:35.476067 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee44f15-d01b-4a59-8b89-dd6d965239d2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9b1c2622934c5185f3b5f1ea623bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:29:35.476722 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee44f15-d01b-4a59-8b89-dd6d965239d2 service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:29:35.477006 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee44f15-d01b-4a59-8b89-dd6d965239d2 service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:29:35.477282 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee44f15-d01b-4a59-8b89-dd6d965239d2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:29:35.478940 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-fee44f15-d01b-4a59-8b89-dd6d965239d2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:29:35.589260 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee44f15-d01b-4a59-8b89-dd6d965239d2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:29:35.590885 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee44f15-d01b-4a59-8b89-dd6d965239d2 service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:29:35.610936 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 702/1419] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:29:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:29:36.587360 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033e2d9c-3e8e-4d8d-89b7-c972e2ace044 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:29:36.587971 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033e2d9c-3e8e-4d8d-89b7-c972e2ace044 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:29:36.587971 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033e2d9c-3e8e-4d8d-89b7-c972e2ace044 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:29:36.589324 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 718/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 15:29:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:29:36.594235 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2a8781-1462-4d38-b63c-f34e0dd66f18 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:29:36.594673 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2a8781-1462-4d38-b63c-f34e0dd66f18 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:29:36.594778 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2a8781-1462-4d38-b63c-f34e0dd66f18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:29:36.650346 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:29:36.688679 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-0f2a8781-1462-4d38-b63c-f34e0dd66f18 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:29:36.690672 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0f2a8781-1462-4d38-b63c-f34e0dd66f18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:29:36.793840 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 703/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 15:29:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:29:36.945285 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033e2d9c-3e8e-4d8d-89b7-c972e2ace044 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:29:36.946834 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-033e2d9c-3e8e-4d8d-89b7-c972e2ace044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:29:37.098318 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033e2d9c-3e8e-4d8d-89b7-c972e2ace044 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:29:37.115811 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:29:37.222855 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f45d444-8ae5-454a-b47b-aca3419025dc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9b1c2622934c5185f3b5f1ea623bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:29:37.223414 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f45d444-8ae5-454a-b47b-aca3419025dc service nova] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:29:37.223732 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f45d444-8ae5-454a-b47b-aca3419025dc service nova] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:29:37.224011 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f45d444-8ae5-454a-b47b-aca3419025dc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:29:37.225653 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-3f45d444-8ae5-454a-b47b-aca3419025dc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:29:37.327870 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f45d444-8ae5-454a-b47b-aca3419025dc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:29:37.329506 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f45d444-8ae5-454a-b47b-aca3419025dc service nova] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:29:37.352051 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 719/1422] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:29:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:29:38.248927 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8358df-8ea9-4b9a-ba18-bf575a1cbc04 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:29:38.249478 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8358df-8ea9-4b9a-ba18-bf575a1cbc04 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:29:38.249478 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8358df-8ea9-4b9a-ba18-bf575a1cbc04 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:29:38.250793 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 704/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 15:29:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:29:38.256197 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bc2ddd-8720-437b-ae32-363b304fa455 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:29:38.256653 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bc2ddd-8720-437b-ae32-363b304fa455 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:29:38.256757 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bc2ddd-8720-437b-ae32-363b304fa455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:29:38.275553 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:29:38.342960 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-a0bc2ddd-8720-437b-ae32-363b304fa455 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:29:38.344233 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-a0bc2ddd-8720-437b-ae32-363b304fa455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:29:38.457907 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 720/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 15:29:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:29:38.607893 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8358df-8ea9-4b9a-ba18-bf575a1cbc04 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:29:38.609733 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0e8358df-8ea9-4b9a-ba18-bf575a1cbc04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:29:38.731765 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:29:38.769156 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8358df-8ea9-4b9a-ba18-bf575a1cbc04 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:29:38.884150 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1516bdb4-8fbc-4e51-8350-c767647cefe6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9b1c2622934c5185f3b5f1ea623bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:29:38.884727 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1516bdb4-8fbc-4e51-8350-c767647cefe6 service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:29:38.885017 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1516bdb4-8fbc-4e51-8350-c767647cefe6 service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:29:38.885303 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1516bdb4-8fbc-4e51-8350-c767647cefe6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:29:38.887018 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-1516bdb4-8fbc-4e51-8350-c767647cefe6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:29:38.964481 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:29:38.978195 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1516bdb4-8fbc-4e51-8350-c767647cefe6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:29:38.979855 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1516bdb4-8fbc-4e51-8350-c767647cefe6 service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:29:39.000801 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 705/1425] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:29:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:29:39.903800 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a480b8a1-be85-4ee8-b60d-89fd18a639c0 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:29:39.904953 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a480b8a1-be85-4ee8-b60d-89fd18a639c0 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:29:39.904953 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a480b8a1-be85-4ee8-b60d-89fd18a639c0 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:29:39.906402 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 721/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 15:29:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:29:39.911515 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6baa01f-c77a-48b5-a811-4d1235d4a591 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:29:39.911980 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6baa01f-c77a-48b5-a811-4d1235d4a591 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:29:39.912117 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6baa01f-c77a-48b5-a811-4d1235d4a591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:29:39.990167 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-c6baa01f-c77a-48b5-a811-4d1235d4a591 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:29:39.991404 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c6baa01f-c77a-48b5-a811-4d1235d4a591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:29:40.100090 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 706/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 15:29:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:29:40.252520 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a480b8a1-be85-4ee8-b60d-89fd18a639c0 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:29:40.254055 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-a480b8a1-be85-4ee8-b60d-89fd18a639c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:29:40.344860 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:29:40.410725 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a480b8a1-be85-4ee8-b60d-89fd18a639c0 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:29:40.530484 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729a7531-fb4a-4c2c-b81a-6a07df3419a4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9b1c2622934c5185f3b5f1ea623bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:29:40.531060 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729a7531-fb4a-4c2c-b81a-6a07df3419a4 service nova] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:29:40.531351 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729a7531-fb4a-4c2c-b81a-6a07df3419a4 service nova] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:29:40.531659 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729a7531-fb4a-4c2c-b81a-6a07df3419a4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:29:40.533308 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-729a7531-fb4a-4c2c-b81a-6a07df3419a4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:29:40.583509 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:29:40.627109 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729a7531-fb4a-4c2c-b81a-6a07df3419a4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:29:40.629088 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729a7531-fb4a-4c2c-b81a-6a07df3419a4 service nova] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:29:40.649351 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 722/1428] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:29:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:29:41.472620 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670fc89c-e369-4a9c-b70a-3a0fb42b8d7a None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:29:41.473202 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670fc89c-e369-4a9c-b70a-3a0fb42b8d7a None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:29:41.473202 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670fc89c-e369-4a9c-b70a-3a0fb42b8d7a None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:29:41.474574 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 707/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 15:29:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:29:41.479429 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5baea3-a87e-42d4-8040-c60076d4fa76 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:29:41.479856 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5baea3-a87e-42d4-8040-c60076d4fa76 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:29:41.480022 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5baea3-a87e-42d4-8040-c60076d4fa76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:29:41.557769 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-2d5baea3-a87e-42d4-8040-c60076d4fa76 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:29:41.558979 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-2d5baea3-a87e-42d4-8040-c60076d4fa76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:29:41.655673 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 723/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 15:29:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:29:41.810528 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670fc89c-e369-4a9c-b70a-3a0fb42b8d7a None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:29:41.812801 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-670fc89c-e369-4a9c-b70a-3a0fb42b8d7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:29:41.862529 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:29:41.965064 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670fc89c-e369-4a9c-b70a-3a0fb42b8d7a None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:29:42.085792 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cac8b93-8fe6-440f-9511-6ef01604c521 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9b1c2622934c5185f3b5f1ea623bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:29:42.086364 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cac8b93-8fe6-440f-9511-6ef01604c521 service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:29:42.086683 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cac8b93-8fe6-440f-9511-6ef01604c521 service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:29:42.086965 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cac8b93-8fe6-440f-9511-6ef01604c521 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:29:42.088662 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-9cac8b93-8fe6-440f-9511-6ef01604c521 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:29:42.097011 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:29:42.183973 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cac8b93-8fe6-440f-9511-6ef01604c521 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:29:42.185623 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cac8b93-8fe6-440f-9511-6ef01604c521 service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:29:42.203889 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 708/1431] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:29:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 15:29:43.031330 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114ee9f2-87f5-4416-9e7b-0ff7b329efa9 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:29:43.031894 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114ee9f2-87f5-4416-9e7b-0ff7b329efa9 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:29:43.031894 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114ee9f2-87f5-4416-9e7b-0ff7b329efa9 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:29:43.033304 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 724/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 15:29:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:29:43.038410 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff50bc1-ff04-4cd8-b0aa-e2ac368e59f8 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:29:43.038844 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff50bc1-ff04-4cd8-b0aa-e2ac368e59f8 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:29:43.038978 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff50bc1-ff04-4cd8-b0aa-e2ac368e59f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:29:43.115110 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-fff50bc1-ff04-4cd8-b0aa-e2ac368e59f8 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:29:43.116373 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-fff50bc1-ff04-4cd8-b0aa-e2ac368e59f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:29:43.189034 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:29:43.220673 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 709/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 15:29:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:29:43.376366 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114ee9f2-87f5-4416-9e7b-0ff7b329efa9 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:29:43.377905 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-114ee9f2-87f5-4416-9e7b-0ff7b329efa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:29:43.396486 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:29:43.530867 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114ee9f2-87f5-4416-9e7b-0ff7b329efa9 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:29:43.621333 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:29:43.648742 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00d9dbd-b8de-4f39-b6c1-3a9f146010a2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9b1c2622934c5185f3b5f1ea623bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:29:43.649400 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00d9dbd-b8de-4f39-b6c1-3a9f146010a2 service nova] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:29:43.649692 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00d9dbd-b8de-4f39-b6c1-3a9f146010a2 service nova] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:29:43.649976 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00d9dbd-b8de-4f39-b6c1-3a9f146010a2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:29:43.651672 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-d00d9dbd-b8de-4f39-b6c1-3a9f146010a2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:29:43.743012 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00d9dbd-b8de-4f39-b6c1-3a9f146010a2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:29:43.744570 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00d9dbd-b8de-4f39-b6c1-3a9f146010a2 service nova] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:29:43.763265 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 725/1434] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:29:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:29:44.600396 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104f5923-f2cb-4d01-b093-624ff1b0b1f5 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:29:44.600951 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104f5923-f2cb-4d01-b093-624ff1b0b1f5 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:29:44.600951 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104f5923-f2cb-4d01-b093-624ff1b0b1f5 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:29:44.602350 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 710/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 15:29:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:29:44.608191 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ec5721-5414-426b-8635-51c306622004 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:29:44.608662 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ec5721-5414-426b-8635-51c306622004 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:29:44.608777 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ec5721-5414-426b-8635-51c306622004 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:29:44.693769 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-d7ec5721-5414-426b-8635-51c306622004 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:29:44.694979 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-d7ec5721-5414-426b-8635-51c306622004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:29:44.742279 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:29:44.806710 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 726/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 15:29:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 15:29:44.962182 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104f5923-f2cb-4d01-b093-624ff1b0b1f5 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:29:44.963747 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-104f5923-f2cb-4d01-b093-624ff1b0b1f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:29:45.124491 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104f5923-f2cb-4d01-b093-624ff1b0b1f5 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:29:45.187118 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:29:45.249341 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e73d0e-00c1-4781-a33a-c80e79c4a402 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9b1c2622934c5185f3b5f1ea623bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:29:45.250000 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e73d0e-00c1-4781-a33a-c80e79c4a402 service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:29:45.250284 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e73d0e-00c1-4781-a33a-c80e79c4a402 service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:29:45.250571 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e73d0e-00c1-4781-a33a-c80e79c4a402 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:29:45.252401 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b9e73d0e-00c1-4781-a33a-c80e79c4a402 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:29:45.344307 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e73d0e-00c1-4781-a33a-c80e79c4a402 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:29:45.345913 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e73d0e-00c1-4781-a33a-c80e79c4a402 service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:29:45.365580 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 711/1437] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:29:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:29:46.191088 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89923cc5-2034-49c5-a7fb-fa1b3152d761 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:29:46.191700 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89923cc5-2034-49c5-a7fb-fa1b3152d761 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:29:46.191700 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89923cc5-2034-49c5-a7fb-fa1b3152d761 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:29:46.193076 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 727/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 15:29:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:29:46.198266 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6e4e2f-b8f4-4b0e-8efe-4d0071b633a4 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:29:46.198708 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6e4e2f-b8f4-4b0e-8efe-4d0071b633a4 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:29:46.198815 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6e4e2f-b8f4-4b0e-8efe-4d0071b633a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:29:46.277884 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-4f6e4e2f-b8f4-4b0e-8efe-4d0071b633a4 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:29:46.279039 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-4f6e4e2f-b8f4-4b0e-8efe-4d0071b633a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:29:46.281554 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:29:46.385274 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 712/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 15:29:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:29:46.541206 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89923cc5-2034-49c5-a7fb-fa1b3152d761 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:29:46.542654 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-89923cc5-2034-49c5-a7fb-fa1b3152d761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:29:46.694489 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89923cc5-2034-49c5-a7fb-fa1b3152d761 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:29:46.722690 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:29:46.814074 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e586caa-f6cb-4ab0-b0d5-29d31f8354b7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9b1c2622934c5185f3b5f1ea623bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:29:46.814634 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e586caa-f6cb-4ab0-b0d5-29d31f8354b7 service nova] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:29:46.814922 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e586caa-f6cb-4ab0-b0d5-29d31f8354b7 service nova] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:29:46.815200 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e586caa-f6cb-4ab0-b0d5-29d31f8354b7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:29:46.816965 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-9e586caa-f6cb-4ab0-b0d5-29d31f8354b7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:29:46.913429 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e586caa-f6cb-4ab0-b0d5-29d31f8354b7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:29:46.915058 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e586caa-f6cb-4ab0-b0d5-29d31f8354b7 service nova] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:29:46.936321 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 728/1440] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:29:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:29:48.940383 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52b6c08-06f4-4aea-b2fb-9f5275f1d4fe None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:29:48.940896 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52b6c08-06f4-4aea-b2fb-9f5275f1d4fe None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:29:48.941017 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52b6c08-06f4-4aea-b2fb-9f5275f1d4fe None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:29:48.942324 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 713/1441] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 15:29:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 15:29:48.947889 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1f71a2-02e4-4ae2-a2a1-2564bdd07505 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:29:48.948351 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1f71a2-02e4-4ae2-a2a1-2564bdd07505 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:29:48.948505 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1f71a2-02e4-4ae2-a2a1-2564bdd07505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:29:48.990644 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:29:49.021097 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-9c1f71a2-02e4-4ae2-a2a1-2564bdd07505 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:29:49.022351 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-9c1f71a2-02e4-4ae2-a2a1-2564bdd07505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:29:49.129333 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 729/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 15:29:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 15:29:49.177247 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52b6c08-06f4-4aea-b2fb-9f5275f1d4fe None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:29:49.178803 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c52b6c08-06f4-4aea-b2fb-9f5275f1d4fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:29:49.296377 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52b6c08-06f4-4aea-b2fb-9f5275f1d4fe None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:29:49.330176 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:29:49.375097 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642ffe66-8c2d-49b6-aa21-5ea75bf0c796 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:29:49.375784 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642ffe66-8c2d-49b6-aa21-5ea75bf0c796 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:29:49.376080 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642ffe66-8c2d-49b6-aa21-5ea75bf0c796 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:29:49.376365 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642ffe66-8c2d-49b6-aa21-5ea75bf0c796 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:29:49.378071 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-642ffe66-8c2d-49b6-aa21-5ea75bf0c796 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:29:49.500951 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642ffe66-8c2d-49b6-aa21-5ea75bf0c796 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:29:49.502300 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642ffe66-8c2d-49b6-aa21-5ea75bf0c796 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:29:49.523660 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 714/1443] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:29:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:29:51.889701 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6158b1ab-93ce-49f9-8580-79c4b7d7f046 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:29:51.890325 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6158b1ab-93ce-49f9-8580-79c4b7d7f046 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:29:51.890325 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6158b1ab-93ce-49f9-8580-79c4b7d7f046 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:29:51.891923 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 730/1444] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 15:29:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:29:51.901608 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9566aa49-af2f-483c-a914-fc0c5ff1e01d None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:29:51.902179 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9566aa49-af2f-483c-a914-fc0c5ff1e01d None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:29:51.902179 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9566aa49-af2f-483c-a914-fc0c5ff1e01d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:29:51.970822 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-9566aa49-af2f-483c-a914-fc0c5ff1e01d None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:29:51.972101 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-9566aa49-af2f-483c-a914-fc0c5ff1e01d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:29:51.986515 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:29:52.083214 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 715/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 15:29:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:29:52.124731 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6158b1ab-93ce-49f9-8580-79c4b7d7f046 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:29:52.126326 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-6158b1ab-93ce-49f9-8580-79c4b7d7f046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:29:52.239864 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6158b1ab-93ce-49f9-8580-79c4b7d7f046 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:29:52.252623 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:29:52.318551 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0dbf99-36c2-4dae-ab1f-83454c8773d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:29:52.319207 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0dbf99-36c2-4dae-ab1f-83454c8773d9 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:29:52.319492 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0dbf99-36c2-4dae-ab1f-83454c8773d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:29:52.319806 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0dbf99-36c2-4dae-ab1f-83454c8773d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:29:52.321472 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-4c0dbf99-36c2-4dae-ab1f-83454c8773d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:29:52.444489 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0dbf99-36c2-4dae-ab1f-83454c8773d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:29:52.446721 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0dbf99-36c2-4dae-ab1f-83454c8773d9 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:29:52.470660 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 731/1446] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:29:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 15:29:54.988778 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7998dd-42d8-472b-a43a-83392d49a829 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:29:54.989335 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7998dd-42d8-472b-a43a-83392d49a829 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:29:54.989420 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7998dd-42d8-472b-a43a-83392d49a829 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:29:54.991095 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 716/1447] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 15:29:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 15:29:55.001879 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9abfa9-e62d-4b26-b158-88da5355af55 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:29:55.002950 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9abfa9-e62d-4b26-b158-88da5355af55 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:29:55.004268 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9abfa9-e62d-4b26-b158-88da5355af55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:29:55.056241 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:29:55.075224 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-0b9abfa9-e62d-4b26-b158-88da5355af55 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:29:55.076684 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-0b9abfa9-e62d-4b26-b158-88da5355af55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:29:55.174145 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 732/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 15:29:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:29:55.213578 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7998dd-42d8-472b-a43a-83392d49a829 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:29:55.215114 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ab7998dd-42d8-472b-a43a-83392d49a829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:29:55.332695 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7998dd-42d8-472b-a43a-83392d49a829 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:29:55.376256 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:29:55.412526 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd3e406-3619-4997-9907-e8d9dba670f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:29:55.413188 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd3e406-3619-4997-9907-e8d9dba670f4 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:29:55.413476 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd3e406-3619-4997-9907-e8d9dba670f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:29:55.413756 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd3e406-3619-4997-9907-e8d9dba670f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:29:55.415418 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-bfd3e406-3619-4997-9907-e8d9dba670f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:29:55.530944 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd3e406-3619-4997-9907-e8d9dba670f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:29:55.532324 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd3e406-3619-4997-9907-e8d9dba670f4 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:29:55.551074 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 717/1449] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:29:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:29:57.804507 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084bef5f-b781-4831-9925-21f40b1fa298 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:29:57.805053 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084bef5f-b781-4831-9925-21f40b1fa298 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:29:57.805053 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084bef5f-b781-4831-9925-21f40b1fa298 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:29:57.806470 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 733/1450] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 15:29:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:29:57.812016 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca01850-a3a2-4563-93cb-cf26954444c3 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:29:57.812676 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca01850-a3a2-4563-93cb-cf26954444c3 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:29:57.812676 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca01850-a3a2-4563-93cb-cf26954444c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:29:57.881109 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-3ca01850-a3a2-4563-93cb-cf26954444c3 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:29:57.882335 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-3ca01850-a3a2-4563-93cb-cf26954444c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:29:57.903233 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:29:57.981073 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 718/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 15:29:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:29:58.022840 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084bef5f-b781-4831-9925-21f40b1fa298 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:29:58.024646 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-084bef5f-b781-4831-9925-21f40b1fa298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:29:58.131218 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084bef5f-b781-4831-9925-21f40b1fa298 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:29:58.152013 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:29:58.211177 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23462c5f-a87f-4b18-97af-cb67541d3702 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:29:58.211871 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23462c5f-a87f-4b18-97af-cb67541d3702 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:29:58.212162 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23462c5f-a87f-4b18-97af-cb67541d3702 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:29:58.212450 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23462c5f-a87f-4b18-97af-cb67541d3702 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:29:58.214111 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-23462c5f-a87f-4b18-97af-cb67541d3702 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:29:58.326702 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23462c5f-a87f-4b18-97af-cb67541d3702 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:29:58.328200 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23462c5f-a87f-4b18-97af-cb67541d3702 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:29:58.347269 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 734/1452] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:29:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:30:00.497739 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a74f4ba-c6ac-4726-88c2-f0f9986ee5fb None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:30:00.498379 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a74f4ba-c6ac-4726-88c2-f0f9986ee5fb None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:30:00.498379 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a74f4ba-c6ac-4726-88c2-f0f9986ee5fb None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:30:00.499883 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 719/1453] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 15:30:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:30:00.505094 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf79eb84-e54f-461a-bf07-83eaf135d44d None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:30:00.505505 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf79eb84-e54f-461a-bf07-83eaf135d44d None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:30:00.505656 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf79eb84-e54f-461a-bf07-83eaf135d44d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:30:00.574701 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:30:00.578617 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-cf79eb84-e54f-461a-bf07-83eaf135d44d None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:30:00.580518 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-cf79eb84-e54f-461a-bf07-83eaf135d44d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:30:00.692388 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 735/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 15:30:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:30:00.733628 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a74f4ba-c6ac-4726-88c2-f0f9986ee5fb None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:30:00.735303 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-6a74f4ba-c6ac-4726-88c2-f0f9986ee5fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:30:00.849441 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a74f4ba-c6ac-4726-88c2-f0f9986ee5fb None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:30:00.899502 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:30:00.925170 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0609e4f0-89c2-4540-9a1c-53e6d7f7c491 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:30:00.925823 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0609e4f0-89c2-4540-9a1c-53e6d7f7c491 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:30:00.926113 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0609e4f0-89c2-4540-9a1c-53e6d7f7c491 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:30:00.926387 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0609e4f0-89c2-4540-9a1c-53e6d7f7c491 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:30:00.928086 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0609e4f0-89c2-4540-9a1c-53e6d7f7c491 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:30:01.043102 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0609e4f0-89c2-4540-9a1c-53e6d7f7c491 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:30:01.045331 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0609e4f0-89c2-4540-9a1c-53e6d7f7c491 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:30:01.064073 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 720/1455] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:30:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 15:30:03.179974 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cff8408-d0aa-4de1-95e6-99d781fd7e4d None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:30:03.180559 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cff8408-d0aa-4de1-95e6-99d781fd7e4d None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:30:03.180559 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cff8408-d0aa-4de1-95e6-99d781fd7e4d None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:30:03.181821 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 736/1456] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 15:30:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:30:03.187835 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe918cf-756f-4fea-8e4a-fbd7ca70039f None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:30:03.188311 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe918cf-756f-4fea-8e4a-fbd7ca70039f None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:30:03.188442 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe918cf-756f-4fea-8e4a-fbd7ca70039f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:30:03.256861 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-1fe918cf-756f-4fea-8e4a-fbd7ca70039f None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:30:03.258095 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-1fe918cf-756f-4fea-8e4a-fbd7ca70039f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:30:03.293692 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:30:03.369546 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 721/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 15:30:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:30:03.413815 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cff8408-d0aa-4de1-95e6-99d781fd7e4d None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:30:03.415386 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-7cff8408-d0aa-4de1-95e6-99d781fd7e4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:30:03.525873 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cff8408-d0aa-4de1-95e6-99d781fd7e4d None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:30:03.554959 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:30:03.600301 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852ba66d-eea4-46d7-9273-9d04af1b5ba9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:30:03.600949 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852ba66d-eea4-46d7-9273-9d04af1b5ba9 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:30:03.601233 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852ba66d-eea4-46d7-9273-9d04af1b5ba9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:30:03.601514 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852ba66d-eea4-46d7-9273-9d04af1b5ba9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:30:03.603160 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-852ba66d-eea4-46d7-9273-9d04af1b5ba9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:30:03.714353 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852ba66d-eea4-46d7-9273-9d04af1b5ba9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:30:03.715835 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852ba66d-eea4-46d7-9273-9d04af1b5ba9 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:30:03.736485 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 737/1458] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:30:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:30:05.912428 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdb9aba-c9fa-4cc3-9ef1-6ece52971f98 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:30:05.912976 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdb9aba-c9fa-4cc3-9ef1-6ece52971f98 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:30:05.912976 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdb9aba-c9fa-4cc3-9ef1-6ece52971f98 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:30:05.914373 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 722/1459] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 15:30:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:30:05.920381 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d7cbda-4969-4271-a782-74e9fe4eeacf None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:30:05.920901 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d7cbda-4969-4271-a782-74e9fe4eeacf None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:30:05.921141 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d7cbda-4969-4271-a782-74e9fe4eeacf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:30:05.989850 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-32d7cbda-4969-4271-a782-74e9fe4eeacf None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:30:05.991210 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-32d7cbda-4969-4271-a782-74e9fe4eeacf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:30:06.003035 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:30:06.102834 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 738/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 15:30:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:30:06.148444 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdb9aba-c9fa-4cc3-9ef1-6ece52971f98 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:30:06.150044 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-2cdb9aba-c9fa-4cc3-9ef1-6ece52971f98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:30:06.268811 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdb9aba-c9fa-4cc3-9ef1-6ece52971f98 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:30:06.330490 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:30:06.353971 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25100433-0921-4601-a31a-54cf31f2f7d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:30:06.354633 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25100433-0921-4601-a31a-54cf31f2f7d2 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:30:06.354922 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25100433-0921-4601-a31a-54cf31f2f7d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:30:06.355205 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25100433-0921-4601-a31a-54cf31f2f7d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:30:06.356926 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-25100433-0921-4601-a31a-54cf31f2f7d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:30:06.487499 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25100433-0921-4601-a31a-54cf31f2f7d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:30:06.490139 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25100433-0921-4601-a31a-54cf31f2f7d2 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:30:06.517701 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 723/1461] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:30:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:30:08.750309 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66df870-a51d-462e-8592-1c411f05509f None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:30:08.750939 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66df870-a51d-462e-8592-1c411f05509f None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:30:08.750939 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66df870-a51d-462e-8592-1c411f05509f None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:30:08.752315 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 739/1462] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 15:30:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:30:08.757692 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1074a4-a2ea-426c-9a30-fd686bfd9f88 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:30:08.758121 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1074a4-a2ea-426c-9a30-fd686bfd9f88 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:30:08.758261 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1074a4-a2ea-426c-9a30-fd686bfd9f88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:30:08.825165 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-1f1074a4-a2ea-426c-9a30-fd686bfd9f88 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:30:08.826979 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-1f1074a4-a2ea-426c-9a30-fd686bfd9f88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:30:08.866617 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:30:08.938945 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 724/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 15:30:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 15:30:08.978230 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66df870-a51d-462e-8592-1c411f05509f None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:30:08.979823 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-b66df870-a51d-462e-8592-1c411f05509f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:30:09.088447 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66df870-a51d-462e-8592-1c411f05509f None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:30:09.125004 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:30:09.161636 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47b09f5-9257-4b56-a120-cd7d3fae3968 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:30:09.162294 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47b09f5-9257-4b56-a120-cd7d3fae3968 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:30:09.162575 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47b09f5-9257-4b56-a120-cd7d3fae3968 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:30:09.162853 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47b09f5-9257-4b56-a120-cd7d3fae3968 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:30:09.164548 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-a47b09f5-9257-4b56-a120-cd7d3fae3968 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:30:09.280926 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47b09f5-9257-4b56-a120-cd7d3fae3968 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:30:09.282410 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47b09f5-9257-4b56-a120-cd7d3fae3968 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:30:09.311368 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 740/1464] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:30:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:30:11.387979 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267eb5aa-c2bc-4474-904c-0b3b6fa0ebe9 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:30:11.388649 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267eb5aa-c2bc-4474-904c-0b3b6fa0ebe9 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:30:11.388649 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267eb5aa-c2bc-4474-904c-0b3b6fa0ebe9 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:30:11.389888 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 725/1465] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 15:30:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:30:11.396216 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a010b5b-8315-413b-bd22-ed7b85daa3a6 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:30:11.396660 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a010b5b-8315-413b-bd22-ed7b85daa3a6 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:30:11.397157 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a010b5b-8315-413b-bd22-ed7b85daa3a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:30:11.466198 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-6a010b5b-8315-413b-bd22-ed7b85daa3a6 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:30:11.467788 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-6a010b5b-8315-413b-bd22-ed7b85daa3a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:30:11.486415 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:30:11.567511 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 741/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 15:30:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 15:30:11.604738 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267eb5aa-c2bc-4474-904c-0b3b6fa0ebe9 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:30:11.606281 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-267eb5aa-c2bc-4474-904c-0b3b6fa0ebe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:30:11.714788 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267eb5aa-c2bc-4474-904c-0b3b6fa0ebe9 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:30:11.784985 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:30:11.792509 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830718a6-ab25-4b3f-bb1a-9887226ccd1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:30:11.793160 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830718a6-ab25-4b3f-bb1a-9887226ccd1f service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:30:11.793447 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830718a6-ab25-4b3f-bb1a-9887226ccd1f service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:30:11.793722 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830718a6-ab25-4b3f-bb1a-9887226ccd1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:30:11.795366 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-830718a6-ab25-4b3f-bb1a-9887226ccd1f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:30:11.907551 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830718a6-ab25-4b3f-bb1a-9887226ccd1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:30:11.909020 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830718a6-ab25-4b3f-bb1a-9887226ccd1f service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:30:11.926577 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 726/1467] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:30:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:30:14.012532 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667cca65-066c-42a2-931a-b307e6b407ee None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:30:14.013054 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667cca65-066c-42a2-931a-b307e6b407ee None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:30:14.013161 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667cca65-066c-42a2-931a-b307e6b407ee None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:30:14.014774 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 742/1468] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 15:30:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:30:14.020339 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d1a281-d268-4af2-a253-b6543671a932 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:30:14.020786 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d1a281-d268-4af2-a253-b6543671a932 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:30:14.020901 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d1a281-d268-4af2-a253-b6543671a932 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:30:14.085893 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-f8d1a281-d268-4af2-a253-b6543671a932 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:30:14.087111 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f8d1a281-d268-4af2-a253-b6543671a932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:30:14.136621 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:30:14.186897 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 727/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 15:30:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:30:14.224445 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667cca65-066c-42a2-931a-b307e6b407ee None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:30:14.225985 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-667cca65-066c-42a2-931a-b307e6b407ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:30:14.330245 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667cca65-066c-42a2-931a-b307e6b407ee None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:30:14.379765 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:30:14.406186 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541def8d-064b-4851-a482-ce13264d2adb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:30:14.406919 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541def8d-064b-4851-a482-ce13264d2adb service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:30:14.407206 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541def8d-064b-4851-a482-ce13264d2adb service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:30:14.407483 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541def8d-064b-4851-a482-ce13264d2adb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:30:14.409166 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-541def8d-064b-4851-a482-ce13264d2adb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:30:14.520704 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541def8d-064b-4851-a482-ce13264d2adb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:30:14.522150 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541def8d-064b-4851-a482-ce13264d2adb service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:30:14.541249 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 743/1470] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:30:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:30:24.185006 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=22 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:30:24.538797 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=66 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:30:35.115416 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d032bd-3a97-498c-aa83-0f1148e900b7 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:30:35.115986 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d032bd-3a97-498c-aa83-0f1148e900b7 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:30:35.116093 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d032bd-3a97-498c-aa83-0f1148e900b7 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:30:35.117435 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 728/1471] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:30:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:30:35.122867 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c6614c-070f-4afa-8682-b63d9fb3d918 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:30:35.124020 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c6614c-070f-4afa-8682-b63d9fb3d918 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:30:35.124333 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c6614c-070f-4afa-8682-b63d9fb3d918 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:30:35.193310 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-00c6614c-070f-4afa-8682-b63d9fb3d918 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:30:35.194512 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-00c6614c-070f-4afa-8682-b63d9fb3d918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:30:35.300680 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 744/1472] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:30:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:30:35.390721 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f99ca7-ff5e-4ca9-b79f-0e758fe02a2d None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:30:35.391013 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f99ca7-ff5e-4ca9-b79f-0e758fe02a2d None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:30:35.391295 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f99ca7-ff5e-4ca9-b79f-0e758fe02a2d None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:30:35.392772 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 729/1473] 10.222.0.99 () {54 vars in 962 bytes} [Thu Jul 14 15:30:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:30:35.397941 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ae405d-b463-4fe0-a56d-183802b854b9 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:30:35.398237 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ae405d-b463-4fe0-a56d-183802b854b9 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:30:35.398523 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ae405d-b463-4fe0-a56d-183802b854b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:30:35.486559 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-b6ae405d-b463-4fe0-a56d-183802b854b9 None None] MFA Rules not processed for user `605e230d50ce44629f554c95b2f79ead`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:30:35.487913 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-b6ae405d-b463-4fe0-a56d-183802b854b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:30:35.510425 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:30:35.572746 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 745/1474] 10.222.0.99 () {58 vars in 1059 bytes} [Thu Jul 14 15:30:35 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:30:35.580133 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f99ca7-ff5e-4ca9-b79f-0e758fe02a2d None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:30:35.581670 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e8f99ca7-ff5e-4ca9-b79f-0e758fe02a2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:30:35.700372 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f99ca7-ff5e-4ca9-b79f-0e758fe02a2d None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:30:35.784850 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89523c9-e0d9-4b8f-aa0f-2dd19059ffa7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605e230d50ce44629f554c95b2f79ead', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:30:35.785443 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89523c9-e0d9-4b8f-aa0f-2dd19059ffa7 service glance] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:30:35.785732 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89523c9-e0d9-4b8f-aa0f-2dd19059ffa7 service glance] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:30:35.786015 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89523c9-e0d9-4b8f-aa0f-2dd19059ffa7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:30:35.787796 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b89523c9-e0d9-4b8f-aa0f-2dd19059ffa7 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:30:35.845716 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:30:35.904463 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89523c9-e0d9-4b8f-aa0f-2dd19059ffa7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:30:35.906069 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89523c9-e0d9-4b8f-aa0f-2dd19059ffa7 service glance] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:30:35.925971 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 730/1475] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:30:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:30:38.199250 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c33376e-ed6a-4208-ac06-97fcd92822c6 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:30:38.199805 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c33376e-ed6a-4208-ac06-97fcd92822c6 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:30:38.199916 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c33376e-ed6a-4208-ac06-97fcd92822c6 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:30:38.201225 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 746/1476] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:30:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:30:38.207518 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c02d9ce-2e1c-4077-bc48-992df15ef7f0 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:30:38.208325 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c02d9ce-2e1c-4077-bc48-992df15ef7f0 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:30:38.208771 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c02d9ce-2e1c-4077-bc48-992df15ef7f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:30:38.277537 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-8c02d9ce-2e1c-4077-bc48-992df15ef7f0 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:30:38.278679 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-8c02d9ce-2e1c-4077-bc48-992df15ef7f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:30:38.383080 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 731/1477] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:30:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:30:38.408662 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c33376e-ed6a-4208-ac06-97fcd92822c6 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:30:38.411115 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-9c33376e-ed6a-4208-ac06-97fcd92822c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:30:38.566380 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c33376e-ed6a-4208-ac06-97fcd92822c6 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:30:38.621874 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:30:38.675901 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2852621-7015-4f21-b355-5fcedc97a0fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:30:38.676569 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2852621-7015-4f21-b355-5fcedc97a0fd admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:30:38.676858 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2852621-7015-4f21-b355-5fcedc97a0fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:30:38.677158 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2852621-7015-4f21-b355-5fcedc97a0fd admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:30:38.677898 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2852621-7015-4f21-b355-5fcedc97a0fd admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:30:38.679866 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: WARNING py.warnings [None req-a2852621-7015-4f21-b355-5fcedc97a0fd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 15:30:38.679866 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: warnings.warn(msg) Jul 14 15:30:38.679866 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: Jul 14 15:30:38.681016 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2852621-7015-4f21-b355-5fcedc97a0fd admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:30:38.701005 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 747/1478] 10.222.0.99 () {56 vars in 1108 bytes} [Thu Jul 14 15:30:38 2022] GET /identity/v3/projects => generated 2553 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:30:40.890196 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf62a15-b9f6-40e4-8dd2-f0fbd9f098db None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:30:40.890921 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf62a15-b9f6-40e4-8dd2-f0fbd9f098db None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:30:40.890921 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf62a15-b9f6-40e4-8dd2-f0fbd9f098db None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:30:40.892295 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 732/1479] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:30:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:30:40.898000 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdb8a52-dc61-4d26-8287-b9a2b09269fe None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:30:40.898407 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdb8a52-dc61-4d26-8287-b9a2b09269fe None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:30:40.898603 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdb8a52-dc61-4d26-8287-b9a2b09269fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:30:40.969068 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-dfdb8a52-dc61-4d26-8287-b9a2b09269fe None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:30:40.970302 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-dfdb8a52-dc61-4d26-8287-b9a2b09269fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:30:41.075084 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 748/1480] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:30:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:30:41.119432 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf62a15-b9f6-40e4-8dd2-f0fbd9f098db None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:30:41.121036 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-4bf62a15-b9f6-40e4-8dd2-f0fbd9f098db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:30:41.131782 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:30:41.303748 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf62a15-b9f6-40e4-8dd2-f0fbd9f098db None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:30:41.386659 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:30:41.420691 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e18a6a2-6629-4b9a-be6b-c7c315b2e4a1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9b1c2622934c5185f3b5f1ea623bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:30:41.421360 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e18a6a2-6629-4b9a-be6b-c7c315b2e4a1 service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:30:41.421646 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e18a6a2-6629-4b9a-be6b-c7c315b2e4a1 service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:30:41.421922 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e18a6a2-6629-4b9a-be6b-c7c315b2e4a1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:30:41.423576 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-8e18a6a2-6629-4b9a-be6b-c7c315b2e4a1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:30:41.514763 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e18a6a2-6629-4b9a-be6b-c7c315b2e4a1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:30:41.516402 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e18a6a2-6629-4b9a-be6b-c7c315b2e4a1 service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:30:41.537159 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 733/1481] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:30:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:30:43.819749 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e918ef-61cb-43cb-9185-12021a0ebcfc None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:30:43.820342 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e918ef-61cb-43cb-9185-12021a0ebcfc None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:30:43.820342 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e918ef-61cb-43cb-9185-12021a0ebcfc None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:30:43.821683 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 749/1482] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:30:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:30:43.829269 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d4df9a-1f1f-4c66-9c2a-ba2936524acf None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:30:43.829886 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d4df9a-1f1f-4c66-9c2a-ba2936524acf None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:30:43.829886 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d4df9a-1f1f-4c66-9c2a-ba2936524acf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:30:43.897964 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-88d4df9a-1f1f-4c66-9c2a-ba2936524acf None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:30:43.899189 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-88d4df9a-1f1f-4c66-9c2a-ba2936524acf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:30:43.946981 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:30:44.006852 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 734/1483] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:30:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 15:30:44.044341 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e918ef-61cb-43cb-9185-12021a0ebcfc None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:30:44.046564 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-c0e918ef-61cb-43cb-9185-12021a0ebcfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:30:44.072972 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:30:44.163228 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e918ef-61cb-43cb-9185-12021a0ebcfc None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:30:44.239029 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b07f9b6-8860-42ed-be4b-f43d83806008 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:30:44.239716 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b07f9b6-8860-42ed-be4b-f43d83806008 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:30:44.240005 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b07f9b6-8860-42ed-be4b-f43d83806008 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:30:44.240288 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b07f9b6-8860-42ed-be4b-f43d83806008 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:30:44.241944 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-1b07f9b6-8860-42ed-be4b-f43d83806008 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:30:44.321212 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:30:44.357536 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b07f9b6-8860-42ed-be4b-f43d83806008 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:30:44.359841 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b07f9b6-8860-42ed-be4b-f43d83806008 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:30:44.379306 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 750/1484] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:30:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:30:46.546848 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1404dbca-26bf-4e24-a5e9-a6851341d49e None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:30:46.547418 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1404dbca-26bf-4e24-a5e9-a6851341d49e None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:30:46.547418 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1404dbca-26bf-4e24-a5e9-a6851341d49e None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:30:46.548872 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 735/1485] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:30:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:30:46.554113 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455cb60d-84fc-43cf-8cf6-e6fb89486066 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:30:46.554535 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455cb60d-84fc-43cf-8cf6-e6fb89486066 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:30:46.554678 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455cb60d-84fc-43cf-8cf6-e6fb89486066 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:30:46.620413 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-455cb60d-84fc-43cf-8cf6-e6fb89486066 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:30:46.621627 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-455cb60d-84fc-43cf-8cf6-e6fb89486066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:30:46.721250 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 751/1486] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:30:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:30:46.763020 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1404dbca-26bf-4e24-a5e9-a6851341d49e None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:30:46.765188 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-1404dbca-26bf-4e24-a5e9-a6851341d49e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:30:46.870539 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1404dbca-26bf-4e24-a5e9-a6851341d49e None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:30:46.943199 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:30:46.944988 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59449be-8587-4f9c-b935-8f13e65be606 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:30:46.945653 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59449be-8587-4f9c-b935-8f13e65be606 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:30:46.946198 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59449be-8587-4f9c-b935-8f13e65be606 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:30:46.946557 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59449be-8587-4f9c-b935-8f13e65be606 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:30:46.948647 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f59449be-8587-4f9c-b935-8f13e65be606 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:30:47.065423 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59449be-8587-4f9c-b935-8f13e65be606 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:30:47.066880 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59449be-8587-4f9c-b935-8f13e65be606 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:30:47.085993 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 736/1487] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:30:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:30:49.245231 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db60cc34-8540-41d4-aedf-e350432c398d None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:30:49.245762 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db60cc34-8540-41d4-aedf-e350432c398d None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:30:49.245762 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db60cc34-8540-41d4-aedf-e350432c398d None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:30:49.247117 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 752/1488] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:30:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:30:49.252565 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fe77e2-86c5-4ca8-b2f9-b3610fffb76d None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:30:49.253020 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fe77e2-86c5-4ca8-b2f9-b3610fffb76d None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:30:49.253168 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fe77e2-86c5-4ca8-b2f9-b3610fffb76d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:30:49.320838 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-29fe77e2-86c5-4ca8-b2f9-b3610fffb76d None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:30:49.322076 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-29fe77e2-86c5-4ca8-b2f9-b3610fffb76d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:30:49.377440 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:30:49.424070 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 737/1489] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:30:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:30:49.466807 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db60cc34-8540-41d4-aedf-e350432c398d None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:30:49.468593 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-db60cc34-8540-41d4-aedf-e350432c398d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:30:49.506448 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:30:49.578628 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db60cc34-8540-41d4-aedf-e350432c398d None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:30:49.654235 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6cc611-c8f4-4095-883a-5fe7008d5adb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:30:49.655837 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6cc611-c8f4-4095-883a-5fe7008d5adb service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:30:49.656133 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6cc611-c8f4-4095-883a-5fe7008d5adb service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:30:49.656416 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6cc611-c8f4-4095-883a-5fe7008d5adb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:30:49.658111 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-9e6cc611-c8f4-4095-883a-5fe7008d5adb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:30:49.745269 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:30:49.778151 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6cc611-c8f4-4095-883a-5fe7008d5adb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:30:49.780292 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6cc611-c8f4-4095-883a-5fe7008d5adb service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:30:49.798596 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 753/1490] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:30:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:30:52.148963 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cb777c-6f75-44b6-bf19-fd646b5a15c7 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:30:52.149532 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cb777c-6f75-44b6-bf19-fd646b5a15c7 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:30:52.149532 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cb777c-6f75-44b6-bf19-fd646b5a15c7 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:30:52.150923 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 738/1491] 10.222.0.99 () {54 vars in 926 bytes} [Thu Jul 14 15:30:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:30:52.156787 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2795bb-b6ad-4e52-9c62-3c8bc1baf156 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:30:52.157432 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2795bb-b6ad-4e52-9c62-3c8bc1baf156 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:30:52.157432 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2795bb-b6ad-4e52-9c62-3c8bc1baf156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:30:52.225848 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-de2795bb-b6ad-4e52-9c62-3c8bc1baf156 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:30:52.227089 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-de2795bb-b6ad-4e52-9c62-3c8bc1baf156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:30:52.335512 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 754/1492] 10.222.0.99 () {58 vars in 1023 bytes} [Thu Jul 14 15:30:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:30:52.375794 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cb777c-6f75-44b6-bf19-fd646b5a15c7 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:30:52.377354 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a9cb777c-6f75-44b6-bf19-fd646b5a15c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:30:52.490376 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cb777c-6f75-44b6-bf19-fd646b5a15c7 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:30:52.566799 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869890bd-2236-4a43-9dbc-a0dd6b0d6dc9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:30:52.567448 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869890bd-2236-4a43-9dbc-a0dd6b0d6dc9 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:30:52.567769 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869890bd-2236-4a43-9dbc-a0dd6b0d6dc9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:30:52.568049 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869890bd-2236-4a43-9dbc-a0dd6b0d6dc9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:30:52.569718 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-869890bd-2236-4a43-9dbc-a0dd6b0d6dc9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:30:52.580360 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:30:52.815919 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869890bd-2236-4a43-9dbc-a0dd6b0d6dc9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:30:52.817378 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869890bd-2236-4a43-9dbc-a0dd6b0d6dc9 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:30:52.835867 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 739/1493] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:30:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:31:02.332549 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=82 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:31:02.833364 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=50 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:32:21.646403 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cdec62-d363-4375-b9d0-9277b7c80b7d None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:32:21.647289 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cdec62-d363-4375-b9d0-9277b7c80b7d None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:32:21.647289 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cdec62-d363-4375-b9d0-9277b7c80b7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:32:21.721048 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-28cdec62-d363-4375-b9d0-9277b7c80b7d None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:32:21.722343 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-28cdec62-d363-4375-b9d0-9277b7c80b7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:32:21.822358 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 755/1494] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:32:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:32:21.831257 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7eed28-3ea4-4b7a-b607-45dbadbef982 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:32:21.831801 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7eed28-3ea4-4b7a-b607-45dbadbef982 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:32:21.831906 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7eed28-3ea4-4b7a-b607-45dbadbef982 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:32:21.908576 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-0b7eed28-3ea4-4b7a-b607-45dbadbef982 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:32:21.909695 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0b7eed28-3ea4-4b7a-b607-45dbadbef982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:32:22.007691 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 740/1495] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:32:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:32:22.013053 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cdec62-d363-4375-b9d0-9277b7c80b7d None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:32:22.014942 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-28cdec62-d363-4375-b9d0-9277b7c80b7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:32:22.026497 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cdec62-d363-4375-b9d0-9277b7c80b7d None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:32:22.089508 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:32:22.125573 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401cac73-46cf-4f0e-81d2-cd865d591365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:32:22.126241 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401cac73-46cf-4f0e-81d2-cd865d591365 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:32:22.126525 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401cac73-46cf-4f0e-81d2-cd865d591365 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:32:22.126800 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401cac73-46cf-4f0e-81d2-cd865d591365 admin admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:32:22.127522 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401cac73-46cf-4f0e-81d2-cd865d591365 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:32:22.127877 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401cac73-46cf-4f0e-81d2-cd865d591365 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:32:22.128927 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: WARNING py.warnings [None req-401cac73-46cf-4f0e-81d2-cd865d591365 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 14 15:32:22.128927 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: warnings.warn(msg) Jul 14 15:32:22.128927 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: Jul 14 15:32:22.129807 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401cac73-46cf-4f0e-81d2-cd865d591365 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:32:22.141676 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 756/1496] 10.222.0.99 () {58 vars in 1152 bytes} [Thu Jul 14 15:32:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:32:22.150085 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7eed28-3ea4-4b7a-b607-45dbadbef982 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:32:22.160274 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7eed28-3ea4-4b7a-b607-45dbadbef982 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:32:22.225165 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:32:22.258525 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c92153e-77aa-410e-b458-1b6335c76c48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:32:22.259185 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c92153e-77aa-410e-b458-1b6335c76c48 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:32:22.259469 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c92153e-77aa-410e-b458-1b6335c76c48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:32:22.259786 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c92153e-77aa-410e-b458-1b6335c76c48 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:32:22.260469 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c92153e-77aa-410e-b458-1b6335c76c48 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:32:22.262221 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: WARNING py.warnings [None req-0c92153e-77aa-410e-b458-1b6335c76c48 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 15:32:22.262221 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: warnings.warn(msg) Jul 14 15:32:22.262221 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: Jul 14 15:32:22.263405 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c92153e-77aa-410e-b458-1b6335c76c48 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:32:22.317462 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 741/1497] 10.222.0.99 () {60 vars in 1152 bytes} [Thu Jul 14 15:32:22 2022] POST /identity/v3/projects => generated 379 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 15:32:22.321848 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401cac73-46cf-4f0e-81d2-cd865d591365 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:32:22.323384 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-401cac73-46cf-4f0e-81d2-cd865d591365 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:32:22.358389 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401cac73-46cf-4f0e-81d2-cd865d591365 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:32:22.464115 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248c4a80-bc3b-417e-ad26-4f42c146b16e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:32:22.464709 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248c4a80-bc3b-417e-ad26-4f42c146b16e admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:32:22.464997 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248c4a80-bc3b-417e-ad26-4f42c146b16e admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:32:22.465280 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248c4a80-bc3b-417e-ad26-4f42c146b16e admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:32:22.465980 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248c4a80-bc3b-417e-ad26-4f42c146b16e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:32:22.467089 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: WARNING py.warnings [None req-248c4a80-bc3b-417e-ad26-4f42c146b16e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 15:32:22.467089 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: warnings.warn(msg) Jul 14 15:32:22.467089 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: Jul 14 15:32:22.468339 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248c4a80-bc3b-417e-ad26-4f42c146b16e admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:32:22.509944 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 757/1498] 10.222.0.99 () {60 vars in 1143 bytes} [Thu Jul 14 15:32:22 2022] POST /identity/v3/users => generated 290 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:32:22.514178 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c92153e-77aa-410e-b458-1b6335c76c48 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:32:22.515780 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0c92153e-77aa-410e-b458-1b6335c76c48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:32:22.578139 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c92153e-77aa-410e-b458-1b6335c76c48 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:32:22.664819 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:32:22.677491 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ced8fda-9241-4c88-a8d8-a2e77de3484f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:32:22.678165 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ced8fda-9241-4c88-a8d8-a2e77de3484f admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:32:22.678454 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ced8fda-9241-4c88-a8d8-a2e77de3484f admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:32:22.678734 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ced8fda-9241-4c88-a8d8-a2e77de3484f admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:32:22.679467 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ced8fda-9241-4c88-a8d8-a2e77de3484f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:32:22.680542 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: WARNING py.warnings [None req-3ced8fda-9241-4c88-a8d8-a2e77de3484f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 14 15:32:22.680542 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: warnings.warn(msg) Jul 14 15:32:22.680542 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: Jul 14 15:32:22.681448 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ced8fda-9241-4c88-a8d8-a2e77de3484f admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:32:22.693915 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 742/1499] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:32:22 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:32:22.698000 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248c4a80-bc3b-417e-ad26-4f42c146b16e admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:32:22.699528 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-248c4a80-bc3b-417e-ad26-4f42c146b16e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:32:22.711563 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248c4a80-bc3b-417e-ad26-4f42c146b16e admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:32:22.738512 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:32:22.810221 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6de6be3-1a44-4f93-948d-0fa033eec3f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:32:22.810894 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6de6be3-1a44-4f93-948d-0fa033eec3f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:32:22.811180 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6de6be3-1a44-4f93-948d-0fa033eec3f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:32:22.811461 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6de6be3-1a44-4f93-948d-0fa033eec3f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:32:22.812231 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6de6be3-1a44-4f93-948d-0fa033eec3f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:32:22.813268 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: WARNING py.warnings [None req-f6de6be3-1a44-4f93-948d-0fa033eec3f5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 14 15:32:22.813268 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: warnings.warn(msg) Jul 14 15:32:22.813268 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: Jul 14 15:32:22.814165 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6de6be3-1a44-4f93-948d-0fa033eec3f5 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:32:22.827178 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 758/1500] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:32:22 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:32:22.830950 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ced8fda-9241-4c88-a8d8-a2e77de3484f admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:32:22.832538 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-3ced8fda-9241-4c88-a8d8-a2e77de3484f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:32:22.867067 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ced8fda-9241-4c88-a8d8-a2e77de3484f admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:32:22.961998 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651b6356-d50d-4696-874b-7928926a8c65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:32:22.962886 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651b6356-d50d-4696-874b-7928926a8c65 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:32:22.963170 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651b6356-d50d-4696-874b-7928926a8c65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:32:22.963446 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651b6356-d50d-4696-874b-7928926a8c65 admin admin] PATH_INFO: `/v3/projects/e557e573928e49c898e3311469940ac1/users/4fac085ebbb8438baf71d2064577322f/roles/b8c98fddc0fd4436b577fd1d3361bc38` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:32:22.986049 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651b6356-d50d-4696-874b-7928926a8c65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e557e573928e49c898e3311469940ac1, user_id=4fac085ebbb8438baf71d2064577322f, role_id=b8c98fddc0fd4436b577fd1d3361bc38)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:32:22.987097 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: WARNING py.warnings [None req-651b6356-d50d-4696-874b-7928926a8c65 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 15:32:22.987097 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: warnings.warn(msg) Jul 14 15:32:22.987097 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: Jul 14 15:32:22.989144 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651b6356-d50d-4696-874b-7928926a8c65 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:32:23.015449 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 743/1501] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 15:32:22 2022] PUT /identity/v3/projects/e557e573928e49c898e3311469940ac1/users/4fac085ebbb8438baf71d2064577322f/roles/b8c98fddc0fd4436b577fd1d3361bc38 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:32:23.027891 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d68565-5eb7-4e4b-925e-f7cf4e36b596 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:32:23.028181 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d68565-5eb7-4e4b-925e-f7cf4e36b596 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:32:23.028463 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d68565-5eb7-4e4b-925e-f7cf4e36b596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:32:23.083751 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-11d68565-5eb7-4e4b-925e-f7cf4e36b596 None None] MFA Rules not processed for user `4fac085ebbb8438baf71d2064577322f`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:32:23.084960 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-11d68565-5eb7-4e4b-925e-f7cf4e36b596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:32:23.164052 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 759/1502] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:32:23 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:32:23.177539 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89f3510-220a-4abe-a27d-e2bc72c3c0b9 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:32:23.177838 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89f3510-220a-4abe-a27d-e2bc72c3c0b9 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:32:23.178124 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89f3510-220a-4abe-a27d-e2bc72c3c0b9 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:32:23.179562 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 744/1503] 10.222.0.99 () {54 vars in 963 bytes} [Thu Jul 14 15:32:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:32:23.184984 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d7c184-2ab6-4a67-9628-43dc238ace6c None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:32:23.185493 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d7c184-2ab6-4a67-9628-43dc238ace6c None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:32:23.185695 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d7c184-2ab6-4a67-9628-43dc238ace6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:32:23.217897 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:32:23.262528 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-65d7c184-2ab6-4a67-9628-43dc238ace6c None None] MFA Rules not processed for user `a3e1a2b83f3d4d88ba5084dfaffd22f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:32:23.264346 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-65d7c184-2ab6-4a67-9628-43dc238ace6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:32:23.339855 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 760/1504] 10.222.0.99 () {58 vars in 1060 bytes} [Thu Jul 14 15:32:23 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:32:23.347613 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89f3510-220a-4abe-a27d-e2bc72c3c0b9 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:32:23.349175 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a89f3510-220a-4abe-a27d-e2bc72c3c0b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:32:23.399534 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:32:23.456042 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89f3510-220a-4abe-a27d-e2bc72c3c0b9 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:32:23.532259 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbeff1b-bfac-4f0a-b0d1-b2a0fac8048e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3e1a2b83f3d4d88ba5084dfaffd22f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:32:23.532925 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbeff1b-bfac-4f0a-b0d1-b2a0fac8048e service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:32:23.533216 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbeff1b-bfac-4f0a-b0d1-b2a0fac8048e service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:32:23.533498 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbeff1b-bfac-4f0a-b0d1-b2a0fac8048e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:32:23.535174 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-afbeff1b-bfac-4f0a-b0d1-b2a0fac8048e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:32:23.645404 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbeff1b-bfac-4f0a-b0d1-b2a0fac8048e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:32:23.647113 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbeff1b-bfac-4f0a-b0d1-b2a0fac8048e service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:32:23.650655 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:32:23.671319 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 745/1505] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:32:23 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:32:23.713548 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ed32a8-ff5a-468b-a53d-6d61a7444042 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:32:23.713977 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ed32a8-ff5a-468b-a53d-6d61a7444042 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:32:23.714120 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ed32a8-ff5a-468b-a53d-6d61a7444042 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:32:23.715550 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 761/1506] 10.222.0.99 () {52 vars in 829 bytes} [Thu Jul 14 15:32:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:32:23.719944 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbeff1b-bfac-4f0a-b0d1-b2a0fac8048e service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:32:23.721189 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-afbeff1b-bfac-4f0a-b0d1-b2a0fac8048e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:32:23.820173 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbeff1b-bfac-4f0a-b0d1-b2a0fac8048e service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:32:23.915347 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3f538a-4e6d-471d-bc22-bdccdf1d6066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:32:23.916239 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3f538a-4e6d-471d-bc22-bdccdf1d6066 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:32:23.916647 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3f538a-4e6d-471d-bc22-bdccdf1d6066 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:32:23.916937 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3f538a-4e6d-471d-bc22-bdccdf1d6066 admin admin] PATH_INFO: `/v3/users/4fac085ebbb8438baf71d2064577322f` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:32:23.930287 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3f538a-4e6d-471d-bc22-bdccdf1d6066 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4fac085ebbb8438baf71d2064577322f)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:32:23.931321 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: WARNING py.warnings [None req-1b3f538a-4e6d-471d-bc22-bdccdf1d6066 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 15:32:23.931321 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: warnings.warn(msg) Jul 14 15:32:23.931321 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: Jul 14 15:32:23.932602 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3f538a-4e6d-471d-bc22-bdccdf1d6066 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:32:23.942311 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:32:23.997304 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-1b3f538a-4e6d-471d-bc22-bdccdf1d6066 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4fac085ebbb8438baf71d2064577322f'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:32:24.007440 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-1b3f538a-4e6d-471d-bc22-bdccdf1d6066 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4fac085ebbb8438baf71d2064577322f'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:32:24.026456 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-1b3f538a-4e6d-471d-bc22-bdccdf1d6066 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4fac085ebbb8438baf71d2064577322f'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:32:24.026916 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-1b3f538a-4e6d-471d-bc22-bdccdf1d6066 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4fac085ebbb8438baf71d2064577322f'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:32:24.027429 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-1b3f538a-4e6d-471d-bc22-bdccdf1d6066 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4fac085ebbb8438baf71d2064577322f'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:32:24.044769 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 746/1507] 10.222.0.99 () {58 vars in 1223 bytes} [Thu Jul 14 15:32:23 2022] DELETE /identity/v3/users/4fac085ebbb8438baf71d2064577322f => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:32:24.049666 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113191d5-5ed0-4345-b756-30a0fea1822e None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:32:24.049966 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113191d5-5ed0-4345-b756-30a0fea1822e None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:32:24.050254 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113191d5-5ed0-4345-b756-30a0fea1822e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:32:24.106128 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-113191d5-5ed0-4345-b756-30a0fea1822e None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:32:24.107350 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-113191d5-5ed0-4345-b756-30a0fea1822e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:32:24.187415 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:32:24.201930 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 762/1508] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:32:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:32:24.211772 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3f538a-4e6d-471d-bc22-bdccdf1d6066 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:32:24.213096 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-1b3f538a-4e6d-471d-bc22-bdccdf1d6066 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:32:24.317958 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3f538a-4e6d-471d-bc22-bdccdf1d6066 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:32:24.382654 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:32:24.406926 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312e6e2a-e551-4bf9-bc17-4f2eda017431 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:32:24.407581 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312e6e2a-e551-4bf9-bc17-4f2eda017431 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:32:24.407895 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312e6e2a-e551-4bf9-bc17-4f2eda017431 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:32:24.408169 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312e6e2a-e551-4bf9-bc17-4f2eda017431 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:32:24.409849 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-312e6e2a-e551-4bf9-bc17-4f2eda017431 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:32:24.522195 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312e6e2a-e551-4bf9-bc17-4f2eda017431 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:32:24.523760 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312e6e2a-e551-4bf9-bc17-4f2eda017431 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:32:24.543219 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 747/1509] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:32:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:32:24.895380 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113191d5-5ed0-4345-b756-30a0fea1822e None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:32:24.897313 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-113191d5-5ed0-4345-b756-30a0fea1822e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:32:24.909891 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113191d5-5ed0-4345-b756-30a0fea1822e None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:32:25.012161 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f64f62b-2cc9-42da-aab3-2898970c61b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:32:25.013025 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f64f62b-2cc9-42da-aab3-2898970c61b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:32:25.013839 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f64f62b-2cc9-42da-aab3-2898970c61b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:32:25.014116 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f64f62b-2cc9-42da-aab3-2898970c61b4 admin admin] PATH_INFO: `/v3/projects/e557e573928e49c898e3311469940ac1` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:32:25.024294 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f64f62b-2cc9-42da-aab3-2898970c61b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e557e573928e49c898e3311469940ac1)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:32:25.025321 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: WARNING py.warnings [None req-4f64f62b-2cc9-42da-aab3-2898970c61b4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 15:32:25.025321 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: warnings.warn(msg) Jul 14 15:32:25.025321 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: Jul 14 15:32:25.026510 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f64f62b-2cc9-42da-aab3-2898970c61b4 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:32:25.084126 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-4f64f62b-2cc9-42da-aab3-2898970c61b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e557e573928e49c898e3311469940ac1', 'request_id': 'req-4f64f62b-2cc9-42da-aab3-2898970c61b4'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:32:25.097918 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-4f64f62b-2cc9-42da-aab3-2898970c61b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e557e573928e49c898e3311469940ac1', 'request_id': 'req-4f64f62b-2cc9-42da-aab3-2898970c61b4'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:32:25.103858 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-4f64f62b-2cc9-42da-aab3-2898970c61b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e557e573928e49c898e3311469940ac1', 'request_id': 'req-4f64f62b-2cc9-42da-aab3-2898970c61b4'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:32:25.129819 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-4f64f62b-2cc9-42da-aab3-2898970c61b4 admin admin] The token cache is being invalidate because project e557e573928e49c898e3311469940ac1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 15:32:25.130111 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-4f64f62b-2cc9-42da-aab3-2898970c61b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:32:25.130659 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-4f64f62b-2cc9-42da-aab3-2898970c61b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:32:25.132079 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 763/1510] 10.222.0.99 () {58 vars in 1232 bytes} [Thu Jul 14 15:32:24 2022] DELETE /identity/v3/projects/e557e573928e49c898e3311469940ac1 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:32:34.540911 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=62 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:32:35.127740 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:18.506093 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed6d830-db5e-4722-8fde-e4168482d5b5 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:18.506707 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed6d830-db5e-4722-8fde-e4168482d5b5 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:18.506817 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed6d830-db5e-4722-8fde-e4168482d5b5 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:18.508223 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 748/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 15:34:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:34:18.513715 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc02fe41-eccd-4c8a-9359-3015d447285c None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:18.514179 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc02fe41-eccd-4c8a-9359-3015d447285c None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:18.514298 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc02fe41-eccd-4c8a-9359-3015d447285c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:18.593965 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-cc02fe41-eccd-4c8a-9359-3015d447285c None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:34:18.595181 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-cc02fe41-eccd-4c8a-9359-3015d447285c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:34:18.699764 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 764/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 15:34:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:34:18.854287 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed6d830-db5e-4722-8fde-e4168482d5b5 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:18.857073 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-fed6d830-db5e-4722-8fde-e4168482d5b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:34:18.977204 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed6d830-db5e-4722-8fde-e4168482d5b5 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:18.987080 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4135b0c2-bfe0-42f7-b20f-cbf4de7e6b87 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9b1c2622934c5185f3b5f1ea623bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:18.987784 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4135b0c2-bfe0-42f7-b20f-cbf4de7e6b87 service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:18.988077 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4135b0c2-bfe0-42f7-b20f-cbf4de7e6b87 service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:18.988361 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4135b0c2-bfe0-42f7-b20f-cbf4de7e6b87 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:18.998095 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4135b0c2-bfe0-42f7-b20f-cbf4de7e6b87 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:18.999732 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4135b0c2-bfe0-42f7-b20f-cbf4de7e6b87 service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:19.009895 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 749/1513] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:34:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:34:19.867149 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d75e043-2137-4733-8e31-65a77a148a18 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:19.867950 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d75e043-2137-4733-8e31-65a77a148a18 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:19.867950 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d75e043-2137-4733-8e31-65a77a148a18 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:19.869138 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 765/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 15:34:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:34:19.874108 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c95c41-3b6f-44f1-987b-0f667e1240bb None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:19.874556 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c95c41-3b6f-44f1-987b-0f667e1240bb None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:19.874666 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c95c41-3b6f-44f1-987b-0f667e1240bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:19.893633 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-b7c95c41-3b6f-44f1-987b-0f667e1240bb None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:34:19.894844 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b7c95c41-3b6f-44f1-987b-0f667e1240bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:34:19.903695 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 750/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 15:34:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 15:34:20.055891 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d75e043-2137-4733-8e31-65a77a148a18 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:20.070508 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d75e043-2137-4733-8e31-65a77a148a18 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:20.076031 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd2660c-8de2-4877-b798-0aab08d38183 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9b1c2622934c5185f3b5f1ea623bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:20.076678 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd2660c-8de2-4877-b798-0aab08d38183 service nova] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:20.076963 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd2660c-8de2-4877-b798-0aab08d38183 service nova] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:20.077240 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd2660c-8de2-4877-b798-0aab08d38183 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:20.085199 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd2660c-8de2-4877-b798-0aab08d38183 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:20.086910 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd2660c-8de2-4877-b798-0aab08d38183 service nova] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:20.097091 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 766/1516] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:34:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:34:22.481679 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdea273f-bfdd-48f9-b635-d87af56e2766 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:22.482264 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdea273f-bfdd-48f9-b635-d87af56e2766 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:22.482264 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdea273f-bfdd-48f9-b635-d87af56e2766 None None] PATH_INFO: `/v3` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:22.484226 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 751/1517] 10.222.0.124 () {54 vars in 926 bytes} [Thu Jul 14 15:34:22 2022] GET /identity/v3 => generated 255 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:22.491212 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237c9106-5809-4973-a502-93714660eed8 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:22.491681 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237c9106-5809-4973-a502-93714660eed8 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:22.491812 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237c9106-5809-4973-a502-93714660eed8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:22.549718 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-237c9106-5809-4973-a502-93714660eed8 None None] MFA Rules not processed for user `dc157c485f794e968a60d8405c9310ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:34:22.550933 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-237c9106-5809-4973-a502-93714660eed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:34:22.567619 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:22.621995 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 767/1518] 10.222.0.124 () {58 vars in 1016 bytes} [Thu Jul 14 15:34:22 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:34:27.207682 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96641026-4973-4ef5-a6df-18617c2465cb None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:27.208220 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96641026-4973-4ef5-a6df-18617c2465cb None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:27.208220 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96641026-4973-4ef5-a6df-18617c2465cb None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:27.209596 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 752/1519] 10.222.0.99 () {54 vars in 960 bytes} [Thu Jul 14 15:34:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:34:27.214848 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1d320c-6c5a-4b6a-8202-86ae7b2dbb71 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:27.215298 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1d320c-6c5a-4b6a-8202-86ae7b2dbb71 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:27.215416 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1d320c-6c5a-4b6a-8202-86ae7b2dbb71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:27.234802 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-ee1d320c-6c5a-4b6a-8202-86ae7b2dbb71 None None] MFA Rules not processed for user `dc157c485f794e968a60d8405c9310ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:34:27.236025 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-ee1d320c-6c5a-4b6a-8202-86ae7b2dbb71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:34:27.244582 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 768/1520] 10.222.0.99 () {58 vars in 1057 bytes} [Thu Jul 14 15:34:27 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:34:27.252267 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96641026-4973-4ef5-a6df-18617c2465cb None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:27.266171 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96641026-4973-4ef5-a6df-18617c2465cb None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:27.271390 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad97206-63d2-4df1-9c8e-8f9148d90d81 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc157c485f794e968a60d8405c9310ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:27.272062 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad97206-63d2-4df1-9c8e-8f9148d90d81 service placement] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:27.272352 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad97206-63d2-4df1-9c8e-8f9148d90d81 service placement] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:27.272633 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad97206-63d2-4df1-9c8e-8f9148d90d81 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:27.281880 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad97206-63d2-4df1-9c8e-8f9148d90d81 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:27.283469 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad97206-63d2-4df1-9c8e-8f9148d90d81 service placement] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:27.293313 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 753/1521] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:34:27 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:34:31.339426 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2465e20-4a81-4000-8dd9-e1c282e334ed None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:31.339995 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2465e20-4a81-4000-8dd9-e1c282e334ed None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:31.339995 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2465e20-4a81-4000-8dd9-e1c282e334ed None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:31.341334 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 769/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 15:34:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:34:31.346632 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025f41fb-3279-4356-8a45-cdf3d00876f8 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:31.347053 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025f41fb-3279-4356-8a45-cdf3d00876f8 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:31.347301 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025f41fb-3279-4356-8a45-cdf3d00876f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:31.365926 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-025f41fb-3279-4356-8a45-cdf3d00876f8 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:34:31.367143 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-025f41fb-3279-4356-8a45-cdf3d00876f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:34:31.375348 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 754/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 15:34:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:34:31.539463 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2465e20-4a81-4000-8dd9-e1c282e334ed None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:31.555875 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2465e20-4a81-4000-8dd9-e1c282e334ed None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:31.562144 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11e3b14-5f40-46f2-88c7-ef89cc1688f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9b1c2622934c5185f3b5f1ea623bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:31.562786 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11e3b14-5f40-46f2-88c7-ef89cc1688f8 service nova] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:31.563077 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11e3b14-5f40-46f2-88c7-ef89cc1688f8 service nova] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:31.563361 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11e3b14-5f40-46f2-88c7-ef89cc1688f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:31.571984 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11e3b14-5f40-46f2-88c7-ef89cc1688f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:31.573569 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11e3b14-5f40-46f2-88c7-ef89cc1688f8 service nova] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:31.583805 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 770/1524] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:34:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:34:32.421796 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f96e64c-a39a-4d59-9cb1-9ab81609762f None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:32.422342 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f96e64c-a39a-4d59-9cb1-9ab81609762f None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:32.422342 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f96e64c-a39a-4d59-9cb1-9ab81609762f None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:32.423758 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 755/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 15:34:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:34:32.428519 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d526809-c998-4eba-90ab-a7bde5d2b955 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:32.428945 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d526809-c998-4eba-90ab-a7bde5d2b955 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:32.429082 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d526809-c998-4eba-90ab-a7bde5d2b955 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:32.448219 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-4d526809-c998-4eba-90ab-a7bde5d2b955 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:34:32.449465 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-4d526809-c998-4eba-90ab-a7bde5d2b955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:34:32.458108 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 771/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 15:34:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:34:32.616040 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f96e64c-a39a-4d59-9cb1-9ab81609762f None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:32.631040 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f96e64c-a39a-4d59-9cb1-9ab81609762f None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:32.636448 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c9a1c5-ef44-429d-9ce3-2cbdff30513b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9b1c2622934c5185f3b5f1ea623bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:32.637096 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c9a1c5-ef44-429d-9ce3-2cbdff30513b service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:32.637389 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c9a1c5-ef44-429d-9ce3-2cbdff30513b service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:32.637670 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c9a1c5-ef44-429d-9ce3-2cbdff30513b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:32.646609 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c9a1c5-ef44-429d-9ce3-2cbdff30513b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:32.648249 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c9a1c5-ef44-429d-9ce3-2cbdff30513b service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:32.657694 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 756/1527] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:34:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:34:36.679963 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c37fd2-8972-4cac-9070-9ffdfab1324a None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:36.680710 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c37fd2-8972-4cac-9070-9ffdfab1324a None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:36.680710 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c37fd2-8972-4cac-9070-9ffdfab1324a None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:36.681922 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 772/1528] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 15:34:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:34:36.687541 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af490d4-8ad7-4a6c-8f7c-66610c2cba71 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:36.688037 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af490d4-8ad7-4a6c-8f7c-66610c2cba71 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:36.688143 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af490d4-8ad7-4a6c-8f7c-66610c2cba71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:36.705966 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-9af490d4-8ad7-4a6c-8f7c-66610c2cba71 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:34:36.707179 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-9af490d4-8ad7-4a6c-8f7c-66610c2cba71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:34:36.715419 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 757/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 15:34:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:34:36.793605 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c37fd2-8972-4cac-9070-9ffdfab1324a None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:36.796078 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-96c37fd2-8972-4cac-9070-9ffdfab1324a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:34:36.876104 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c37fd2-8972-4cac-9070-9ffdfab1324a None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:36.879676 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dac6438-d2cd-4dc5-addf-ca7cbc2fa060 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:36.880331 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dac6438-d2cd-4dc5-addf-ca7cbc2fa060 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:36.880614 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dac6438-d2cd-4dc5-addf-ca7cbc2fa060 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:36.880889 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dac6438-d2cd-4dc5-addf-ca7cbc2fa060 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:36.890404 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dac6438-d2cd-4dc5-addf-ca7cbc2fa060 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:36.891892 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dac6438-d2cd-4dc5-addf-ca7cbc2fa060 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:36.902138 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 773/1530] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:34:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:34:38.994011 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5b2a7d-d4e5-449a-98df-91a9849427e6 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:38.994549 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5b2a7d-d4e5-449a-98df-91a9849427e6 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:38.994549 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5b2a7d-d4e5-449a-98df-91a9849427e6 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:38.995995 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 758/1531] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 15:34:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:34:39.002188 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0ed6a5-ea83-4256-88ac-5067a0d074fb None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:39.002684 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0ed6a5-ea83-4256-88ac-5067a0d074fb None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:39.002789 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0ed6a5-ea83-4256-88ac-5067a0d074fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:39.023369 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-ff0ed6a5-ea83-4256-88ac-5067a0d074fb None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:34:39.024630 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-ff0ed6a5-ea83-4256-88ac-5067a0d074fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:34:39.033149 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 774/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 15:34:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:34:39.069803 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5b2a7d-d4e5-449a-98df-91a9849427e6 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:39.082561 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5b2a7d-d4e5-449a-98df-91a9849427e6 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:39.086436 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e2b189-e0f7-4934-8701-c9ed02e22176 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:39.087144 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e2b189-e0f7-4934-8701-c9ed02e22176 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:39.087369 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e2b189-e0f7-4934-8701-c9ed02e22176 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:39.087676 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e2b189-e0f7-4934-8701-c9ed02e22176 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:39.098187 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e2b189-e0f7-4934-8701-c9ed02e22176 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:39.099653 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e2b189-e0f7-4934-8701-c9ed02e22176 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:39.110335 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 759/1533] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:34:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:34:42.878963 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9ed429-338e-4ffa-baf6-134512dfa3ff None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:42.879549 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9ed429-338e-4ffa-baf6-134512dfa3ff None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:42.879549 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9ed429-338e-4ffa-baf6-134512dfa3ff None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:42.880950 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 775/1534] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 15:34:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:34:42.886305 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5207f937-dc1d-4e32-b5ca-de8eb5d27688 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:42.886725 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5207f937-dc1d-4e32-b5ca-de8eb5d27688 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:42.886881 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5207f937-dc1d-4e32-b5ca-de8eb5d27688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:42.905831 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-5207f937-dc1d-4e32-b5ca-de8eb5d27688 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:34:42.907038 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-5207f937-dc1d-4e32-b5ca-de8eb5d27688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:34:42.914843 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 760/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 15:34:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:34:43.222444 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9ed429-338e-4ffa-baf6-134512dfa3ff None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:43.224213 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-9f9ed429-338e-4ffa-baf6-134512dfa3ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:34:43.290437 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:43.304575 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9ed429-338e-4ffa-baf6-134512dfa3ff None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:43.308170 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96142be9-ac6b-4256-a0ad-b1edab336949 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3e1a2b83f3d4d88ba5084dfaffd22f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:43.308815 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96142be9-ac6b-4256-a0ad-b1edab336949 service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:43.309102 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96142be9-ac6b-4256-a0ad-b1edab336949 service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:43.309389 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96142be9-ac6b-4256-a0ad-b1edab336949 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:43.319224 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96142be9-ac6b-4256-a0ad-b1edab336949 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:43.320702 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96142be9-ac6b-4256-a0ad-b1edab336949 service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:43.330336 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 776/1536] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:34:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:34:45.437075 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2e36fe-4401-4185-a91b-16c265ef7e9f None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:45.437649 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2e36fe-4401-4185-a91b-16c265ef7e9f None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:45.437649 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2e36fe-4401-4185-a91b-16c265ef7e9f None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:45.439035 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 761/1537] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 15:34:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:34:45.445016 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb105630-c95a-47aa-ae61-b629fd84a2ec None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:45.445493 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb105630-c95a-47aa-ae61-b629fd84a2ec None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:45.445615 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb105630-c95a-47aa-ae61-b629fd84a2ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:45.469725 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-fb105630-c95a-47aa-ae61-b629fd84a2ec None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:34:45.470962 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-fb105630-c95a-47aa-ae61-b629fd84a2ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:34:45.479514 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 777/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 15:34:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:34:45.789790 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2e36fe-4401-4185-a91b-16c265ef7e9f None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:45.791484 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-dd2e36fe-4401-4185-a91b-16c265ef7e9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:34:45.799306 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=84 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:45.805581 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2e36fe-4401-4185-a91b-16c265ef7e9f None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:45.809075 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc83113-fc50-4d7d-b9b4-0d331118ce2c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3e1a2b83f3d4d88ba5084dfaffd22f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:45.809718 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc83113-fc50-4d7d-b9b4-0d331118ce2c service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:45.810006 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc83113-fc50-4d7d-b9b4-0d331118ce2c service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:45.810291 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc83113-fc50-4d7d-b9b4-0d331118ce2c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:45.820041 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc83113-fc50-4d7d-b9b4-0d331118ce2c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:45.821448 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc83113-fc50-4d7d-b9b4-0d331118ce2c service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:45.832097 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 762/1539] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:34:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:34:55.463571 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=11 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:55.826983 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=5 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:24.364209 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e09657-2b0e-42d8-a0ab-478c902b4e5c None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:24.364872 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e09657-2b0e-42d8-a0ab-478c902b4e5c None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:24.364872 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e09657-2b0e-42d8-a0ab-478c902b4e5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:24.390076 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-39e09657-2b0e-42d8-a0ab-478c902b4e5c None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:36:24.391346 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-39e09657-2b0e-42d8-a0ab-478c902b4e5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:24.405862 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 778/1540] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:36:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:36:24.416379 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be0291b-c5ca-4046-8195-d9c9ab27837f None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:24.416986 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be0291b-c5ca-4046-8195-d9c9ab27837f None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:24.416986 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be0291b-c5ca-4046-8195-d9c9ab27837f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:24.444219 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-8be0291b-c5ca-4046-8195-d9c9ab27837f None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:36:24.450165 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-8be0291b-c5ca-4046-8195-d9c9ab27837f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:24.462848 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 763/1541] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:36:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:36:24.469310 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e09657-2b0e-42d8-a0ab-478c902b4e5c None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:24.481808 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e09657-2b0e-42d8-a0ab-478c902b4e5c None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:24.490956 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1643d638-a690-4243-a440-0cc0e260b41d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:24.491669 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1643d638-a690-4243-a440-0cc0e260b41d admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:24.491964 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1643d638-a690-4243-a440-0cc0e260b41d admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:24.492349 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1643d638-a690-4243-a440-0cc0e260b41d admin admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:24.493106 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1643d638-a690-4243-a440-0cc0e260b41d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:36:24.493423 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1643d638-a690-4243-a440-0cc0e260b41d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:24.495305 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1643d638-a690-4243-a440-0cc0e260b41d admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:24.522620 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 779/1542] 10.222.0.99 () {58 vars in 1152 bytes} [Thu Jul 14 15:36:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:36:24.531840 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be0291b-c5ca-4046-8195-d9c9ab27837f None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:24.544767 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be0291b-c5ca-4046-8195-d9c9ab27837f None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:24.550396 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a367a1-282a-4797-aa2a-f777a2566b68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:24.553450 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a367a1-282a-4797-aa2a-f777a2566b68 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:24.553933 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a367a1-282a-4797-aa2a-f777a2566b68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:24.554156 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a367a1-282a-4797-aa2a-f777a2566b68 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:24.554832 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a367a1-282a-4797-aa2a-f777a2566b68 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:24.561758 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a367a1-282a-4797-aa2a-f777a2566b68 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:24.635271 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 764/1543] 10.222.0.99 () {60 vars in 1152 bytes} [Thu Jul 14 15:36:24 2022] POST /identity/v3/projects => generated 379 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:36:24.639429 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1643d638-a690-4243-a440-0cc0e260b41d admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:24.671652 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1643d638-a690-4243-a440-0cc0e260b41d admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:24.776766 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9a48e1-7863-4c81-be5e-abeebaf4a1c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:24.777473 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9a48e1-7863-4c81-be5e-abeebaf4a1c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:24.777765 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9a48e1-7863-4c81-be5e-abeebaf4a1c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:24.778048 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9a48e1-7863-4c81-be5e-abeebaf4a1c7 admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:24.778763 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9a48e1-7863-4c81-be5e-abeebaf4a1c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:24.781461 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9a48e1-7863-4c81-be5e-abeebaf4a1c7 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:24.818074 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78219ef-b46d-4ff0-8d1d-576866d1caf9 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:24.818437 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78219ef-b46d-4ff0-8d1d-576866d1caf9 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:24.818703 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78219ef-b46d-4ff0-8d1d-576866d1caf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:24.818914 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 780/1544] 10.222.0.99 () {60 vars in 1143 bytes} [Thu Jul 14 15:36:24 2022] POST /identity/v3/users => generated 290 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:36:24.829504 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9a48e1-7863-4c81-be5e-abeebaf4a1c7 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:24.849049 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9a48e1-7863-4c81-be5e-abeebaf4a1c7 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:24.860467 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-a78219ef-b46d-4ff0-8d1d-576866d1caf9 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:36:24.862410 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a78219ef-b46d-4ff0-8d1d-576866d1caf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:24.862613 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd311ff-ebe8-41ff-945d-6653c5089b68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:24.864776 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd311ff-ebe8-41ff-945d-6653c5089b68 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:24.865104 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd311ff-ebe8-41ff-945d-6653c5089b68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:24.866951 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd311ff-ebe8-41ff-945d-6653c5089b68 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:24.868457 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd311ff-ebe8-41ff-945d-6653c5089b68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:24.870320 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd311ff-ebe8-41ff-945d-6653c5089b68 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:24.882018 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 765/1545] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:36:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 15:36:24.888153 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090584e2-2f33-434c-be46-1bb67757d139 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:24.888355 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090584e2-2f33-434c-be46-1bb67757d139 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:24.888649 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090584e2-2f33-434c-be46-1bb67757d139 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:24.901382 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 781/1546] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:36:24 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 15:36:24.908358 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd311ff-ebe8-41ff-945d-6653c5089b68 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:24.926004 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-090584e2-2f33-434c-be46-1bb67757d139 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:36:24.927533 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-090584e2-2f33-434c-be46-1bb67757d139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:24.933523 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd311ff-ebe8-41ff-945d-6653c5089b68 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:24.958177 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 766/1547] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:36:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 15:36:24.959344 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011570d1-019d-4db4-bb6f-f854f679d901 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:24.959617 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011570d1-019d-4db4-bb6f-f854f679d901 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:24.959910 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011570d1-019d-4db4-bb6f-f854f679d901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:24.981849 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cba1792-2375-492e-bee5-1672bbc0ac1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:24.982565 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cba1792-2375-492e-bee5-1672bbc0ac1f admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:24.982844 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cba1792-2375-492e-bee5-1672bbc0ac1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:24.983113 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cba1792-2375-492e-bee5-1672bbc0ac1f admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:24.983891 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cba1792-2375-492e-bee5-1672bbc0ac1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:24.992289 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-011570d1-019d-4db4-bb6f-f854f679d901 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:36:24.993538 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-011570d1-019d-4db4-bb6f-f854f679d901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:24.998557 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cba1792-2375-492e-bee5-1672bbc0ac1f admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:25.010853 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 767/1548] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:36:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:36:25.012233 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a91924-598d-450f-bee8-d0f007a5c611 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:25.012533 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a91924-598d-450f-bee8-d0f007a5c611 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:25.012822 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a91924-598d-450f-bee8-d0f007a5c611 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:25.027590 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 782/1549] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:36:24 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:36:25.029746 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dacebc-4511-42d7-946e-bd15ffcd1200 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:25.030249 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dacebc-4511-42d7-946e-bd15ffcd1200 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:25.030684 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dacebc-4511-42d7-946e-bd15ffcd1200 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:25.048368 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-31a91924-598d-450f-bee8-d0f007a5c611 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:36:25.051110 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-31a91924-598d-450f-bee8-d0f007a5c611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:25.075224 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-b8dacebc-4511-42d7-946e-bd15ffcd1200 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:36:25.076792 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-b8dacebc-4511-42d7-946e-bd15ffcd1200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:25.080618 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 768/1550] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:36:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:36:25.081694 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a91924-598d-450f-bee8-d0f007a5c611 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:25.098129 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a91924-598d-450f-bee8-d0f007a5c611 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:25.105483 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41ff1c0-9bc8-4a03-8295-535a2b403d7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:25.107449 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41ff1c0-9bc8-4a03-8295-535a2b403d7d admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:25.107774 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41ff1c0-9bc8-4a03-8295-535a2b403d7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:25.108066 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41ff1c0-9bc8-4a03-8295-535a2b403d7d admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:25.108866 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41ff1c0-9bc8-4a03-8295-535a2b403d7d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:36:25.109208 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41ff1c0-9bc8-4a03-8295-535a2b403d7d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:25.110256 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: WARNING py.warnings [None req-e41ff1c0-9bc8-4a03-8295-535a2b403d7d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 14 15:36:25.110256 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: warnings.warn(msg) Jul 14 15:36:25.110256 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: Jul 14 15:36:25.111429 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 783/1551] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:36:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:36:25.112774 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dacebc-4511-42d7-946e-bd15ffcd1200 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:25.121437 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41ff1c0-9bc8-4a03-8295-535a2b403d7d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:25.134643 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dacebc-4511-42d7-946e-bd15ffcd1200 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:25.142839 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2767ea42-dae0-4782-8521-164f0c22cc8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:25.144087 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2767ea42-dae0-4782-8521-164f0c22cc8c admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:25.144584 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2767ea42-dae0-4782-8521-164f0c22cc8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:25.145065 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2767ea42-dae0-4782-8521-164f0c22cc8c admin admin] PATH_INFO: `/v3/projects/b9490b542a1f4c78aef9015ecb97dabd/users/3db7ba60a24747efb55601b6e2401983/roles/b8c98fddc0fd4436b577fd1d3361bc38` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:25.158192 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 769/1552] 10.222.0.99 () {58 vars in 1152 bytes} [Thu Jul 14 15:36:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:36:25.159880 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3c726b-a147-4a9d-828d-7b5708739114 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:25.160410 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3c726b-a147-4a9d-828d-7b5708739114 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:25.161429 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3c726b-a147-4a9d-828d-7b5708739114 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:25.180423 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2767ea42-dae0-4782-8521-164f0c22cc8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9490b542a1f4c78aef9015ecb97dabd, user_id=3db7ba60a24747efb55601b6e2401983, role_id=b8c98fddc0fd4436b577fd1d3361bc38)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:25.181488 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: WARNING py.warnings [None req-2767ea42-dae0-4782-8521-164f0c22cc8c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 15:36:25.181488 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: warnings.warn(msg) Jul 14 15:36:25.181488 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: Jul 14 15:36:25.187629 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2767ea42-dae0-4782-8521-164f0c22cc8c admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:25.187843 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-aa3c726b-a147-4a9d-828d-7b5708739114 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:36:25.187843 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-aa3c726b-a147-4a9d-828d-7b5708739114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:25.197491 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 770/1553] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:36:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:36:25.198814 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d186fb2-c9e5-4c7e-b5cb-6403c45aed55 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:25.199108 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d186fb2-c9e5-4c7e-b5cb-6403c45aed55 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:25.199391 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d186fb2-c9e5-4c7e-b5cb-6403c45aed55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:25.204216 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 784/1554] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 15:36:25 2022] PUT /identity/v3/projects/b9490b542a1f4c78aef9015ecb97dabd/users/3db7ba60a24747efb55601b6e2401983/roles/b8c98fddc0fd4436b577fd1d3361bc38 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:36:25.206301 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2767ea42-dae0-4782-8521-164f0c22cc8c admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:25.221182 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-9d186fb2-c9e5-4c7e-b5cb-6403c45aed55 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:36:25.222422 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-9d186fb2-c9e5-4c7e-b5cb-6403c45aed55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:25.283471 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2767ea42-dae0-4782-8521-164f0c22cc8c admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:25.288735 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79789c8-88ef-4a03-9839-19fcc489a805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:25.289304 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79789c8-88ef-4a03-9839-19fcc489a805 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:25.289609 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79789c8-88ef-4a03-9839-19fcc489a805 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:25.289893 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79789c8-88ef-4a03-9839-19fcc489a805 admin admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:25.290788 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79789c8-88ef-4a03-9839-19fcc489a805 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:36:25.291144 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79789c8-88ef-4a03-9839-19fcc489a805 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:25.293087 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79789c8-88ef-4a03-9839-19fcc489a805 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:25.305910 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 771/1555] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:36:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:36:25.307372 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe04011-7fd3-42f6-9829-2e9fc52b0e8e None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:25.307913 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe04011-7fd3-42f6-9829-2e9fc52b0e8e None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:25.308425 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe04011-7fd3-42f6-9829-2e9fc52b0e8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:25.310284 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 785/1556] 10.222.0.99 () {58 vars in 1152 bytes} [Thu Jul 14 15:36:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:36:25.311306 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79789c8-88ef-4a03-9839-19fcc489a805 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:25.324066 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79789c8-88ef-4a03-9839-19fcc489a805 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:25.329334 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913c91d7-033a-489c-a256-6b2e665d4e34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:25.330013 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913c91d7-033a-489c-a256-6b2e665d4e34 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:25.330013 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913c91d7-033a-489c-a256-6b2e665d4e34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:25.330013 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913c91d7-033a-489c-a256-6b2e665d4e34 admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:25.331248 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913c91d7-033a-489c-a256-6b2e665d4e34 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:25.332323 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: WARNING py.warnings [None req-913c91d7-033a-489c-a256-6b2e665d4e34 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 15:36:25.332323 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: warnings.warn(msg) Jul 14 15:36:25.332323 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: Jul 14 15:36:25.333518 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913c91d7-033a-489c-a256-6b2e665d4e34 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:25.337365 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-efe04011-7fd3-42f6-9829-2e9fc52b0e8e None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:36:25.338594 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-efe04011-7fd3-42f6-9829-2e9fc52b0e8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:25.343400 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:25.347868 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 772/1557] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:36:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:36:25.349318 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe04011-7fd3-42f6-9829-2e9fc52b0e8e None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:25.362052 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe04011-7fd3-42f6-9829-2e9fc52b0e8e None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:25.367376 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ce118a-8556-4705-87a9-97cd88fe988b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:25.368196 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ce118a-8556-4705-87a9-97cd88fe988b admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:25.368469 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ce118a-8556-4705-87a9-97cd88fe988b admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:25.368749 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ce118a-8556-4705-87a9-97cd88fe988b admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:25.369483 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ce118a-8556-4705-87a9-97cd88fe988b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:36:25.369790 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ce118a-8556-4705-87a9-97cd88fe988b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:25.375110 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 786/1558] 10.222.0.99 () {60 vars in 1152 bytes} [Thu Jul 14 15:36:25 2022] POST /identity/v3/projects => generated 367 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:36:25.375818 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ce118a-8556-4705-87a9-97cd88fe988b admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:25.375936 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558d6155-842d-409d-83cc-339ff959766d None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:25.375936 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558d6155-842d-409d-83cc-339ff959766d None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:25.375936 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558d6155-842d-409d-83cc-339ff959766d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:25.391262 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-558d6155-842d-409d-83cc-339ff959766d None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:36:25.391692 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 773/1559] 10.222.0.99 () {58 vars in 1152 bytes} [Thu Jul 14 15:36:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:36:25.392706 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-558d6155-842d-409d-83cc-339ff959766d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:25.394074 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ca12cb-b0ec-4868-b629-0466d7c6b05f None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:25.394575 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ca12cb-b0ec-4868-b629-0466d7c6b05f None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:25.394940 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ca12cb-b0ec-4868-b629-0466d7c6b05f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:25.421436 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-21ca12cb-b0ec-4868-b629-0466d7c6b05f None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:36:25.422959 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-21ca12cb-b0ec-4868-b629-0466d7c6b05f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:25.471129 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 787/1560] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:36:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:36:25.472526 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f00c645-b2ee-461e-97ab-9c31c896b4f4 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:25.473080 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f00c645-b2ee-461e-97ab-9c31c896b4f4 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:25.473527 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f00c645-b2ee-461e-97ab-9c31c896b4f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:25.483062 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:25.493552 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-9f00c645-b2ee-461e-97ab-9c31c896b4f4 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:36:25.494514 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 774/1561] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:36:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:36:25.495819 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-9f00c645-b2ee-461e-97ab-9c31c896b4f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:25.495941 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0f34f3-10cb-4bde-bf83-661c452799b2 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:25.496158 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0f34f3-10cb-4bde-bf83-661c452799b2 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:25.496418 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0f34f3-10cb-4bde-bf83-661c452799b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:25.516255 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 788/1562] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:36:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:36:25.517675 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f00c645-b2ee-461e-97ab-9c31c896b4f4 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:25.519498 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-9a0f34f3-10cb-4bde-bf83-661c452799b2 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:36:25.521089 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-9a0f34f3-10cb-4bde-bf83-661c452799b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:25.529408 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 775/1563] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:36:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:36:25.529848 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f00c645-b2ee-461e-97ab-9c31c896b4f4 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:25.530486 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2768cc0a-42b6-44d0-a17f-ad0a33da27e2 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:25.530778 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2768cc0a-42b6-44d0-a17f-ad0a33da27e2 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:25.531066 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2768cc0a-42b6-44d0-a17f-ad0a33da27e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:25.542645 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68afcf00-aec2-45ff-9170-938b052941d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:25.544014 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68afcf00-aec2-45ff-9170-938b052941d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:25.544382 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68afcf00-aec2-45ff-9170-938b052941d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:25.544674 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68afcf00-aec2-45ff-9170-938b052941d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:25.545947 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68afcf00-aec2-45ff-9170-938b052941d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:25.549016 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68afcf00-aec2-45ff-9170-938b052941d5 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:25.554051 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-2768cc0a-42b6-44d0-a17f-ad0a33da27e2 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:36:25.555273 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-2768cc0a-42b6-44d0-a17f-ad0a33da27e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:25.563542 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 776/1564] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:36:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:36:25.564361 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2768cc0a-42b6-44d0-a17f-ad0a33da27e2 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:25.574492 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2768cc0a-42b6-44d0-a17f-ad0a33da27e2 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:25.581975 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 789/1565] 10.222.0.99 () {60 vars in 1152 bytes} [Thu Jul 14 15:36:25 2022] POST /identity/v3/projects => generated 373 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:36:25.581975 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68afcf00-aec2-45ff-9170-938b052941d5 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:25.583526 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81274975-5805-4c38-b999-acd766372cd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:25.584231 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81274975-5805-4c38-b999-acd766372cd1 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:25.584518 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81274975-5805-4c38-b999-acd766372cd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:25.584798 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81274975-5805-4c38-b999-acd766372cd1 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:25.585509 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81274975-5805-4c38-b999-acd766372cd1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:25.586535 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: WARNING py.warnings [None req-81274975-5805-4c38-b999-acd766372cd1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 15:36:25.586535 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: warnings.warn(msg) Jul 14 15:36:25.586535 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: Jul 14 15:36:25.587766 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81274975-5805-4c38-b999-acd766372cd1 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:25.611242 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 777/1566] 10.222.0.99 () {60 vars in 1143 bytes} [Thu Jul 14 15:36:25 2022] POST /identity/v3/users => generated 284 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:36:25.612126 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81274975-5805-4c38-b999-acd766372cd1 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:25.613819 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-81274975-5805-4c38-b999-acd766372cd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:25.651680 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68afcf00-aec2-45ff-9170-938b052941d5 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:25.657055 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4580a8db-da2c-4559-8f13-c6767e69a477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:25.657731 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4580a8db-da2c-4559-8f13-c6767e69a477 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:25.658024 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4580a8db-da2c-4559-8f13-c6767e69a477 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:25.658309 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4580a8db-da2c-4559-8f13-c6767e69a477 admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:25.659108 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4580a8db-da2c-4559-8f13-c6767e69a477 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:25.660972 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4580a8db-da2c-4559-8f13-c6767e69a477 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:25.673870 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81274975-5805-4c38-b999-acd766372cd1 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:25.690310 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 790/1567] 10.222.0.99 () {60 vars in 1152 bytes} [Thu Jul 14 15:36:25 2022] POST /identity/v3/projects => generated 393 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:36:25.691659 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efd11bd-0eaf-441a-b1f9-11329afb16a7 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:25.691956 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efd11bd-0eaf-441a-b1f9-11329afb16a7 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:25.692239 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efd11bd-0eaf-441a-b1f9-11329afb16a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:25.710723 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-3efd11bd-0eaf-441a-b1f9-11329afb16a7 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:36:25.711585 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-3efd11bd-0eaf-441a-b1f9-11329afb16a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:25.725978 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eeaa8e-3543-4265-ae8f-7989137dc68d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:25.726637 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eeaa8e-3543-4265-ae8f-7989137dc68d service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:25.726954 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eeaa8e-3543-4265-ae8f-7989137dc68d service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:25.727238 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eeaa8e-3543-4265-ae8f-7989137dc68d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:25.738238 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98eeaa8e-3543-4265-ae8f-7989137dc68d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:25.739722 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98eeaa8e-3543-4265-ae8f-7989137dc68d service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:25.750941 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 778/1568] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:36:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:36:25.752183 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47108650-4d90-49f4-8361-9e3c8ffb1786 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:25.752502 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47108650-4d90-49f4-8361-9e3c8ffb1786 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:25.752887 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47108650-4d90-49f4-8361-9e3c8ffb1786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:25.759396 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:25.775326 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-47108650-4d90-49f4-8361-9e3c8ffb1786 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:36:25.776887 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-47108650-4d90-49f4-8361-9e3c8ffb1786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:25.800994 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 791/1569] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:36:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:36:25.802716 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efd11bd-0eaf-441a-b1f9-11329afb16a7 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:25.821175 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efd11bd-0eaf-441a-b1f9-11329afb16a7 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:25.830273 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06fcd7d-407f-4e96-981e-ee96bddce933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:25.831393 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06fcd7d-407f-4e96-981e-ee96bddce933 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:25.831963 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06fcd7d-407f-4e96-981e-ee96bddce933 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:25.832864 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06fcd7d-407f-4e96-981e-ee96bddce933 admin admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:25.834003 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06fcd7d-407f-4e96-981e-ee96bddce933 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:36:25.834630 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06fcd7d-407f-4e96-981e-ee96bddce933 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:25.836961 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06fcd7d-407f-4e96-981e-ee96bddce933 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:25.854276 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 792/1570] 10.222.0.99 () {58 vars in 1152 bytes} [Thu Jul 14 15:36:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:36:25.854808 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 779/1571] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:36:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:36:25.855303 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06fcd7d-407f-4e96-981e-ee96bddce933 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:25.855671 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47108650-4d90-49f4-8361-9e3c8ffb1786 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:25.867788 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47108650-4d90-49f4-8361-9e3c8ffb1786 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:25.873232 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9205d6-3f19-4b7f-ab31-52bc06bd9f88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:25.874026 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9205d6-3f19-4b7f-ab31-52bc06bd9f88 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:25.874699 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9205d6-3f19-4b7f-ab31-52bc06bd9f88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:25.876002 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9205d6-3f19-4b7f-ab31-52bc06bd9f88 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:25.876701 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9205d6-3f19-4b7f-ab31-52bc06bd9f88 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:25.879752 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9205d6-3f19-4b7f-ab31-52bc06bd9f88 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:25.883369 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06fcd7d-407f-4e96-981e-ee96bddce933 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:25.889315 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc30ddb1-c4b2-4a8f-9e3e-f2332dc9e357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:25.890320 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc30ddb1-c4b2-4a8f-9e3e-f2332dc9e357 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:25.891457 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc30ddb1-c4b2-4a8f-9e3e-f2332dc9e357 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:25.892185 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc30ddb1-c4b2-4a8f-9e3e-f2332dc9e357 admin admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:25.892764 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc30ddb1-c4b2-4a8f-9e3e-f2332dc9e357 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:36:25.893191 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc30ddb1-c4b2-4a8f-9e3e-f2332dc9e357 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:25.895220 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc30ddb1-c4b2-4a8f-9e3e-f2332dc9e357 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:25.905314 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 780/1572] 10.222.0.99 () {60 vars in 1143 bytes} [Thu Jul 14 15:36:25 2022] POST /identity/v3/users => generated 287 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:36:25.906440 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9205d6-3f19-4b7f-ab31-52bc06bd9f88 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:25.909497 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 793/1573] 10.222.0.99 () {58 vars in 1152 bytes} [Thu Jul 14 15:36:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:36:25.910349 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc30ddb1-c4b2-4a8f-9e3e-f2332dc9e357 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:25.924249 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc30ddb1-c4b2-4a8f-9e3e-f2332dc9e357 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:25.926411 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9205d6-3f19-4b7f-ab31-52bc06bd9f88 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:25.931320 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf0a411-f948-431d-8be5-dd828d8a7b54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:25.932045 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf0a411-f948-431d-8be5-dd828d8a7b54 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:25.932338 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf0a411-f948-431d-8be5-dd828d8a7b54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:25.932624 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf0a411-f948-431d-8be5-dd828d8a7b54 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:25.932825 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ed075b-cdff-4af0-89da-c0735fd7412d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:25.933374 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf0a411-f948-431d-8be5-dd828d8a7b54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:25.933960 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ed075b-cdff-4af0-89da-c0735fd7412d admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:25.934455 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ed075b-cdff-4af0-89da-c0735fd7412d admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:25.934958 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ed075b-cdff-4af0-89da-c0735fd7412d admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:25.935103 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf0a411-f948-431d-8be5-dd828d8a7b54 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:25.936440 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ed075b-cdff-4af0-89da-c0735fd7412d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:25.938677 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:25.939529 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ed075b-cdff-4af0-89da-c0735fd7412d admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:25.952719 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 781/1574] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:36:25 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:36:25.954330 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf0a411-f948-431d-8be5-dd828d8a7b54 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:25.963464 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 794/1575] 10.222.0.99 () {60 vars in 1143 bytes} [Thu Jul 14 15:36:25 2022] POST /identity/v3/users => generated 297 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:36:25.964441 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ed075b-cdff-4af0-89da-c0735fd7412d admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:25.969890 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf0a411-f948-431d-8be5-dd828d8a7b54 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:25.975265 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13e89b3-ae25-45bf-b818-9d565680bb53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:25.975743 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ed075b-cdff-4af0-89da-c0735fd7412d admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:25.976103 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13e89b3-ae25-45bf-b818-9d565680bb53 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:25.976598 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13e89b3-ae25-45bf-b818-9d565680bb53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:25.976893 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13e89b3-ae25-45bf-b818-9d565680bb53 admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:25.977684 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13e89b3-ae25-45bf-b818-9d565680bb53 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:36:25.977919 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13e89b3-ae25-45bf-b818-9d565680bb53 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:25.979540 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13e89b3-ae25-45bf-b818-9d565680bb53 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:25.980616 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fe281d-8b4a-4020-b7dd-4dff1e0a0c5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:25.981302 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fe281d-8b4a-4020-b7dd-4dff1e0a0c5d admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:25.981572 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fe281d-8b4a-4020-b7dd-4dff1e0a0c5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:25.981855 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fe281d-8b4a-4020-b7dd-4dff1e0a0c5d admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:25.982547 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fe281d-8b4a-4020-b7dd-4dff1e0a0c5d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:25.985083 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fe281d-8b4a-4020-b7dd-4dff1e0a0c5d admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:26.002408 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 782/1576] 10.222.0.99 () {58 vars in 1152 bytes} [Thu Jul 14 15:36:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:36:26.002408 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13e89b3-ae25-45bf-b818-9d565680bb53 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:26.015375 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13e89b3-ae25-45bf-b818-9d565680bb53 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:26.022888 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64554c86-4cb0-4da9-916e-63b723bad888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:26.024188 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64554c86-4cb0-4da9-916e-63b723bad888 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:26.024942 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64554c86-4cb0-4da9-916e-63b723bad888 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:26.025519 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64554c86-4cb0-4da9-916e-63b723bad888 admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:26.026477 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64554c86-4cb0-4da9-916e-63b723bad888 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:36:26.026794 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64554c86-4cb0-4da9-916e-63b723bad888 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:26.028716 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64554c86-4cb0-4da9-916e-63b723bad888 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:26.030518 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 795/1577] 10.222.0.99 () {60 vars in 1152 bytes} [Thu Jul 14 15:36:25 2022] POST /identity/v3/projects => generated 375 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:36:26.032460 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fe281d-8b4a-4020-b7dd-4dff1e0a0c5d admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:26.047964 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 783/1578] 10.222.0.99 () {58 vars in 1152 bytes} [Thu Jul 14 15:36:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:36:26.049079 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64554c86-4cb0-4da9-916e-63b723bad888 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:26.069774 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64554c86-4cb0-4da9-916e-63b723bad888 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:26.139555 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fe281d-8b4a-4020-b7dd-4dff1e0a0c5d admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:26.142942 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2104aa49-4982-4811-8b0a-21426a990bb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:26.144561 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2104aa49-4982-4811-8b0a-21426a990bb2 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:26.145043 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2104aa49-4982-4811-8b0a-21426a990bb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:26.145678 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2104aa49-4982-4811-8b0a-21426a990bb2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:26.146734 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2104aa49-4982-4811-8b0a-21426a990bb2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:26.147149 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866c30e9-3f44-45f4-960a-547f0dabce55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:26.148186 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866c30e9-3f44-45f4-960a-547f0dabce55 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:26.148658 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866c30e9-3f44-45f4-960a-547f0dabce55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:26.149103 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866c30e9-3f44-45f4-960a-547f0dabce55 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:26.149248 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2104aa49-4982-4811-8b0a-21426a990bb2 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:26.150220 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866c30e9-3f44-45f4-960a-547f0dabce55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:26.152231 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866c30e9-3f44-45f4-960a-547f0dabce55 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:26.171398 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 796/1579] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:36:26 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:36:26.173778 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866c30e9-3f44-45f4-960a-547f0dabce55 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:26.185981 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866c30e9-3f44-45f4-960a-547f0dabce55 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:26.190837 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 784/1580] 10.222.0.99 () {60 vars in 1152 bytes} [Thu Jul 14 15:36:26 2022] POST /identity/v3/projects => generated 369 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:36:26.194471 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2104aa49-4982-4811-8b0a-21426a990bb2 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:26.196593 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0e9617-32e2-4bf1-a2a7-55b405a26613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:26.197600 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0e9617-32e2-4bf1-a2a7-55b405a26613 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:26.198031 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0e9617-32e2-4bf1-a2a7-55b405a26613 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:26.198322 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0e9617-32e2-4bf1-a2a7-55b405a26613 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:26.199251 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0e9617-32e2-4bf1-a2a7-55b405a26613 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:26.201444 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0e9617-32e2-4bf1-a2a7-55b405a26613 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:26.218090 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 797/1581] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:36:26 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:36:26.219484 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0e9617-32e2-4bf1-a2a7-55b405a26613 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:26.231508 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0e9617-32e2-4bf1-a2a7-55b405a26613 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:26.282275 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2104aa49-4982-4811-8b0a-21426a990bb2 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:26.287789 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5a0831-5c7d-4e22-9a3e-9e60cee3dba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:26.288503 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5a0831-5c7d-4e22-9a3e-9e60cee3dba0 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:26.288957 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5a0831-5c7d-4e22-9a3e-9e60cee3dba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:26.289296 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5a0831-5c7d-4e22-9a3e-9e60cee3dba0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:26.290056 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5a0831-5c7d-4e22-9a3e-9e60cee3dba0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:26.291817 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5a0831-5c7d-4e22-9a3e-9e60cee3dba0 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:26.305462 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 785/1582] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:36:26 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:36:26.306180 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5a0831-5c7d-4e22-9a3e-9e60cee3dba0 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:26.318330 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5a0831-5c7d-4e22-9a3e-9e60cee3dba0 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:26.323201 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d148f717-3bad-4aac-a5b4-f8d3ec84c5d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:26.323956 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d148f717-3bad-4aac-a5b4-f8d3ec84c5d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:26.324244 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d148f717-3bad-4aac-a5b4-f8d3ec84c5d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:26.324519 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d148f717-3bad-4aac-a5b4-f8d3ec84c5d1 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:26.325221 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d148f717-3bad-4aac-a5b4-f8d3ec84c5d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:26.327222 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d148f717-3bad-4aac-a5b4-f8d3ec84c5d1 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:26.354561 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 786/1583] 10.222.0.99 () {60 vars in 1143 bytes} [Thu Jul 14 15:36:26 2022] POST /identity/v3/users => generated 288 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:36:26.354561 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d148f717-3bad-4aac-a5b4-f8d3ec84c5d1 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:26.366144 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d148f717-3bad-4aac-a5b4-f8d3ec84c5d1 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:26.371144 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f318f7ec-a2c6-4363-bbec-a612c6f6e46f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:26.371936 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f318f7ec-a2c6-4363-bbec-a612c6f6e46f admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:26.372225 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f318f7ec-a2c6-4363-bbec-a612c6f6e46f admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:26.372506 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f318f7ec-a2c6-4363-bbec-a612c6f6e46f admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:26.373199 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f318f7ec-a2c6-4363-bbec-a612c6f6e46f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:26.376639 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f318f7ec-a2c6-4363-bbec-a612c6f6e46f admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:26.413093 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 787/1584] 10.222.0.99 () {60 vars in 1152 bytes} [Thu Jul 14 15:36:26 2022] POST /identity/v3/projects => generated 389 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:36:26.413093 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f318f7ec-a2c6-4363-bbec-a612c6f6e46f admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:26.417164 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9976c5-1555-4916-a99f-2cf2c77c7042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:26.417841 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9976c5-1555-4916-a99f-2cf2c77c7042 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:26.418134 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9976c5-1555-4916-a99f-2cf2c77c7042 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:26.418417 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9976c5-1555-4916-a99f-2cf2c77c7042 admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:26.419122 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9976c5-1555-4916-a99f-2cf2c77c7042 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:26.421082 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9976c5-1555-4916-a99f-2cf2c77c7042 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:26.426655 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:26.432395 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:26.458146 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 798/1585] 10.222.0.99 () {60 vars in 1152 bytes} [Thu Jul 14 15:36:26 2022] POST /identity/v3/projects => generated 375 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:36:26.458146 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9976c5-1555-4916-a99f-2cf2c77c7042 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:26.512425 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f318f7ec-a2c6-4363-bbec-a612c6f6e46f admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:26.533419 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9976c5-1555-4916-a99f-2cf2c77c7042 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:26.539890 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47694d2f-8de3-4c58-a5a8-99b50b9873c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:26.540667 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47694d2f-8de3-4c58-a5a8-99b50b9873c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:26.541018 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47694d2f-8de3-4c58-a5a8-99b50b9873c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:26.541542 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47694d2f-8de3-4c58-a5a8-99b50b9873c1 admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:26.542333 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47694d2f-8de3-4c58-a5a8-99b50b9873c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:26.544546 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47694d2f-8de3-4c58-a5a8-99b50b9873c1 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:26.581747 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 799/1586] 10.222.0.99 () {60 vars in 1143 bytes} [Thu Jul 14 15:36:26 2022] POST /identity/v3/users => generated 285 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:36:26.582894 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47694d2f-8de3-4c58-a5a8-99b50b9873c1 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:26.596236 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47694d2f-8de3-4c58-a5a8-99b50b9873c1 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:26.596740 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915bce73-0308-4c24-8f3b-26f6931a434d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:26.597561 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915bce73-0308-4c24-8f3b-26f6931a434d admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:26.597924 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915bce73-0308-4c24-8f3b-26f6931a434d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:26.598274 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915bce73-0308-4c24-8f3b-26f6931a434d admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:26.599098 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915bce73-0308-4c24-8f3b-26f6931a434d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:26.600918 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915bce73-0308-4c24-8f3b-26f6931a434d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:26.602035 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e14717-e8f7-4225-9dff-715d8d77695e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:26.603384 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e14717-e8f7-4225-9dff-715d8d77695e admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:26.604277 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e14717-e8f7-4225-9dff-715d8d77695e admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:26.604660 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e14717-e8f7-4225-9dff-715d8d77695e admin admin] PATH_INFO: `/v3/projects/e7c68922b5dc4b028e3f0d386a3ff8b8/users/b27ce02ec0f5452d80820cf92c4e77ba/roles/b8c98fddc0fd4436b577fd1d3361bc38` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:26.614365 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 788/1587] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:36:26 2022] GET /identity/v3/roles => generated 1420 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:36:26.615379 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915bce73-0308-4c24-8f3b-26f6931a434d admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:26.627945 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915bce73-0308-4c24-8f3b-26f6931a434d admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:26.629509 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e14717-e8f7-4225-9dff-715d8d77695e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7c68922b5dc4b028e3f0d386a3ff8b8, user_id=b27ce02ec0f5452d80820cf92c4e77ba, role_id=b8c98fddc0fd4436b577fd1d3361bc38)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:26.632685 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e14717-e8f7-4225-9dff-715d8d77695e admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:26.633181 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b411411-c966-48d1-8d4a-703aba0e2e7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:26.634439 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b411411-c966-48d1-8d4a-703aba0e2e7d admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:26.634746 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b411411-c966-48d1-8d4a-703aba0e2e7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:26.635062 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b411411-c966-48d1-8d4a-703aba0e2e7d admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:26.636156 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b411411-c966-48d1-8d4a-703aba0e2e7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:26.638155 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b411411-c966-48d1-8d4a-703aba0e2e7d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:26.650286 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 800/1588] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 15:36:26 2022] PUT /identity/v3/projects/e7c68922b5dc4b028e3f0d386a3ff8b8/users/b27ce02ec0f5452d80820cf92c4e77ba/roles/b8c98fddc0fd4436b577fd1d3361bc38 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:36:26.651583 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 789/1589] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:36:26 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:36:26.652068 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e14717-e8f7-4225-9dff-715d8d77695e admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:26.652695 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b411411-c966-48d1-8d4a-703aba0e2e7d admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:26.668873 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b411411-c966-48d1-8d4a-703aba0e2e7d admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:26.732197 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e14717-e8f7-4225-9dff-715d8d77695e admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:26.739024 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed648bbb-aa3d-4a71-8aa7-8f3cc0ee7952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:26.739933 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed648bbb-aa3d-4a71-8aa7-8f3cc0ee7952 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:26.740349 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed648bbb-aa3d-4a71-8aa7-8f3cc0ee7952 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:26.740976 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed648bbb-aa3d-4a71-8aa7-8f3cc0ee7952 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:26.742238 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed648bbb-aa3d-4a71-8aa7-8f3cc0ee7952 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:26.744280 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed648bbb-aa3d-4a71-8aa7-8f3cc0ee7952 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:26.749727 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09160131-b69a-4551-8562-b05a0a784c02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:26.750756 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09160131-b69a-4551-8562-b05a0a784c02 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:26.751897 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09160131-b69a-4551-8562-b05a0a784c02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:26.752366 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09160131-b69a-4551-8562-b05a0a784c02 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:26.754632 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09160131-b69a-4551-8562-b05a0a784c02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:26.757354 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09160131-b69a-4551-8562-b05a0a784c02 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:26.757638 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 801/1590] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:36:26 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:36:26.759434 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed648bbb-aa3d-4a71-8aa7-8f3cc0ee7952 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:26.782440 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed648bbb-aa3d-4a71-8aa7-8f3cc0ee7952 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:26.788419 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6aef25-8078-4cb2-b008-cdb3c13bb98a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:26.789333 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6aef25-8078-4cb2-b008-cdb3c13bb98a admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:26.789632 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6aef25-8078-4cb2-b008-cdb3c13bb98a admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:26.790083 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6aef25-8078-4cb2-b008-cdb3c13bb98a admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:26.790794 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6aef25-8078-4cb2-b008-cdb3c13bb98a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:26.792810 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6aef25-8078-4cb2-b008-cdb3c13bb98a admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:26.797270 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 790/1591] 10.222.0.99 () {60 vars in 1143 bytes} [Thu Jul 14 15:36:26 2022] POST /identity/v3/users => generated 295 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:36:26.798530 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09160131-b69a-4551-8562-b05a0a784c02 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:26.816031 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09160131-b69a-4551-8562-b05a0a784c02 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:26.820062 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 802/1592] 10.222.0.99 () {60 vars in 1143 bytes} [Thu Jul 14 15:36:26 2022] POST /identity/v3/users => generated 288 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:36:26.820062 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6aef25-8078-4cb2-b008-cdb3c13bb98a admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:26.830940 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cb0acf-31f4-4ec2-aa53-5a5cbb14a60a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:26.831796 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cb0acf-31f4-4ec2-aa53-5a5cbb14a60a admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:26.832148 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cb0acf-31f4-4ec2-aa53-5a5cbb14a60a admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:26.832498 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cb0acf-31f4-4ec2-aa53-5a5cbb14a60a admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:26.833596 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cb0acf-31f4-4ec2-aa53-5a5cbb14a60a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:26.835649 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cb0acf-31f4-4ec2-aa53-5a5cbb14a60a admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:26.837540 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6aef25-8078-4cb2-b008-cdb3c13bb98a admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:26.845123 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6af47a6-8265-4f4e-ac7c-844711d75541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:26.846400 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6af47a6-8265-4f4e-ac7c-844711d75541 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:26.846882 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6af47a6-8265-4f4e-ac7c-844711d75541 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:26.847349 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6af47a6-8265-4f4e-ac7c-844711d75541 admin admin] PATH_INFO: `/v3/projects/7e3a8ab2527749e79988c5e02b1ae864/users/47c16ba599394d9cb3e0261f1e89cc80/roles/b8c98fddc0fd4436b577fd1d3361bc38` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:26.848084 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 791/1593] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:36:26 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:36:26.849674 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c3508d-b498-473c-b46f-94bbd4b5b014 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:26.850024 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c3508d-b498-473c-b46f-94bbd4b5b014 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:26.850393 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c3508d-b498-473c-b46f-94bbd4b5b014 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:26.866955 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6af47a6-8265-4f4e-ac7c-844711d75541 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e3a8ab2527749e79988c5e02b1ae864, user_id=47c16ba599394d9cb3e0261f1e89cc80, role_id=b8c98fddc0fd4436b577fd1d3361bc38)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:26.871419 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6af47a6-8265-4f4e-ac7c-844711d75541 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:26.884766 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-12c3508d-b498-473c-b46f-94bbd4b5b014 None None] MFA Rules not processed for user `b27ce02ec0f5452d80820cf92c4e77ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:36:26.886517 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-12c3508d-b498-473c-b46f-94bbd4b5b014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:26.905459 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 803/1594] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 15:36:26 2022] PUT /identity/v3/projects/7e3a8ab2527749e79988c5e02b1ae864/users/47c16ba599394d9cb3e0261f1e89cc80/roles/b8c98fddc0fd4436b577fd1d3361bc38 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:36:26.905459 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6af47a6-8265-4f4e-ac7c-844711d75541 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:26.905758 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:26.912504 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:26.974508 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6af47a6-8265-4f4e-ac7c-844711d75541 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:26.980729 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 792/1595] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:36:26 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:36:26.985289 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c3508d-b498-473c-b46f-94bbd4b5b014 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:26.986215 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fecb33-0ead-4a32-bf0e-083ddbf1833a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:26.987649 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fecb33-0ead-4a32-bf0e-083ddbf1833a admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:26.987649 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fecb33-0ead-4a32-bf0e-083ddbf1833a admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:26.987812 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fecb33-0ead-4a32-bf0e-083ddbf1833a admin admin] PATH_INFO: `/v3/projects/2a542b3ba16943a4ab7d2624d961b4a5/users/d01c301ea20d4acab8dff1148501a975/roles/b8c98fddc0fd4436b577fd1d3361bc38` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:26.999521 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c3508d-b498-473c-b46f-94bbd4b5b014 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:27.005926 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3ea606-0281-43d7-9430-e21224a96ba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:27.007220 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3ea606-0281-43d7-9430-e21224a96ba2 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:27.007534 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3ea606-0281-43d7-9430-e21224a96ba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:27.007849 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3ea606-0281-43d7-9430-e21224a96ba2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:27.008785 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3ea606-0281-43d7-9430-e21224a96ba2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:27.011062 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3ea606-0281-43d7-9430-e21224a96ba2 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:27.012691 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fecb33-0ead-4a32-bf0e-083ddbf1833a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a542b3ba16943a4ab7d2624d961b4a5, user_id=d01c301ea20d4acab8dff1148501a975, role_id=b8c98fddc0fd4436b577fd1d3361bc38)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:27.015456 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fecb33-0ead-4a32-bf0e-083ddbf1833a admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:27.024741 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 793/1596] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:36:26 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:36:27.025776 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3ea606-0281-43d7-9430-e21224a96ba2 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:27.035735 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 804/1597] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 15:36:26 2022] PUT /identity/v3/projects/2a542b3ba16943a4ab7d2624d961b4a5/users/d01c301ea20d4acab8dff1148501a975/roles/b8c98fddc0fd4436b577fd1d3361bc38 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:36:27.036880 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fecb33-0ead-4a32-bf0e-083ddbf1833a admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:27.041884 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3ea606-0281-43d7-9430-e21224a96ba2 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:27.122189 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fecb33-0ead-4a32-bf0e-083ddbf1833a admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:27.127402 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2731fde4-738a-42ac-846e-ff70abf28a0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:27.128237 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2731fde4-738a-42ac-846e-ff70abf28a0e admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:27.128533 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2731fde4-738a-42ac-846e-ff70abf28a0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:27.128655 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03d371b-2fbe-4f57-a32c-a5a791ded23e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:27.128832 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2731fde4-738a-42ac-846e-ff70abf28a0e admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:27.129488 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03d371b-2fbe-4f57-a32c-a5a791ded23e admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:27.129602 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2731fde4-738a-42ac-846e-ff70abf28a0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:27.129874 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03d371b-2fbe-4f57-a32c-a5a791ded23e admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:27.130228 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03d371b-2fbe-4f57-a32c-a5a791ded23e admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:27.131043 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03d371b-2fbe-4f57-a32c-a5a791ded23e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:27.131421 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2731fde4-738a-42ac-846e-ff70abf28a0e admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:27.132959 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03d371b-2fbe-4f57-a32c-a5a791ded23e admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:27.153765 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 794/1598] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:36:27 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:36:27.155072 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2731fde4-738a-42ac-846e-ff70abf28a0e admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:27.159954 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 805/1599] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:36:27 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:36:27.161709 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d1cccc-591a-4401-8336-9e4bf764b331 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:27.162201 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d1cccc-591a-4401-8336-9e4bf764b331 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:27.162543 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d1cccc-591a-4401-8336-9e4bf764b331 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:27.172665 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2731fde4-738a-42ac-846e-ff70abf28a0e admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:27.177878 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391a0678-b376-410f-84bc-f8a66c83dda6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:27.179230 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391a0678-b376-410f-84bc-f8a66c83dda6 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:27.180052 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391a0678-b376-410f-84bc-f8a66c83dda6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:27.180052 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391a0678-b376-410f-84bc-f8a66c83dda6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:27.180575 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391a0678-b376-410f-84bc-f8a66c83dda6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:27.182204 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391a0678-b376-410f-84bc-f8a66c83dda6 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:27.197789 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 795/1600] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:36:27 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:36:27.197990 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-68d1cccc-591a-4401-8336-9e4bf764b331 None None] MFA Rules not processed for user `47c16ba599394d9cb3e0261f1e89cc80`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:36:27.198103 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391a0678-b376-410f-84bc-f8a66c83dda6 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:27.199226 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-68d1cccc-591a-4401-8336-9e4bf764b331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:27.282065 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391a0678-b376-410f-84bc-f8a66c83dda6 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:27.286890 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccabb520-7147-4a90-b48a-c9d84d9f111b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3e1a2b83f3d4d88ba5084dfaffd22f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:27.287747 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccabb520-7147-4a90-b48a-c9d84d9f111b service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:27.288012 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccabb520-7147-4a90-b48a-c9d84d9f111b service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:27.288397 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccabb520-7147-4a90-b48a-c9d84d9f111b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:27.306329 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 806/1601] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:36:27 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:36:27.307350 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d1cccc-591a-4401-8336-9e4bf764b331 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:27.323526 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d1cccc-591a-4401-8336-9e4bf764b331 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:27.330368 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cc5a20-40a4-4b7c-a93f-452907ae2b42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:27.331292 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cc5a20-40a4-4b7c-a93f-452907ae2b42 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:27.331588 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cc5a20-40a4-4b7c-a93f-452907ae2b42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:27.331950 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cc5a20-40a4-4b7c-a93f-452907ae2b42 admin admin] PATH_INFO: `/v3/projects/5172d2ae5b244afd99f04b3cd6e056d0/users/30ffdf00311048a6a47d142e5231119d/roles/b8c98fddc0fd4436b577fd1d3361bc38` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:27.355890 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cc5a20-40a4-4b7c-a93f-452907ae2b42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5172d2ae5b244afd99f04b3cd6e056d0, user_id=30ffdf00311048a6a47d142e5231119d, role_id=b8c98fddc0fd4436b577fd1d3361bc38)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:27.360278 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cc5a20-40a4-4b7c-a93f-452907ae2b42 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:27.361415 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:27.368413 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:27.386577 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccabb520-7147-4a90-b48a-c9d84d9f111b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:27.388095 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccabb520-7147-4a90-b48a-c9d84d9f111b service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:27.395360 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 807/1602] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 15:36:27 2022] PUT /identity/v3/projects/5172d2ae5b244afd99f04b3cd6e056d0/users/30ffdf00311048a6a47d142e5231119d/roles/b8c98fddc0fd4436b577fd1d3361bc38 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:36:27.397418 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2555cdb-74a5-4df3-92cd-427ba72995dc None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:27.397653 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2555cdb-74a5-4df3-92cd-427ba72995dc None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:27.397941 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2555cdb-74a5-4df3-92cd-427ba72995dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:27.399527 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 796/1603] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:36:27 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:36:27.403647 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccabb520-7147-4a90-b48a-c9d84d9f111b service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:27.417076 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccabb520-7147-4a90-b48a-c9d84d9f111b service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:27.436838 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-e2555cdb-74a5-4df3-92cd-427ba72995dc None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:36:27.438881 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-e2555cdb-74a5-4df3-92cd-427ba72995dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:27.522525 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6877675e-a5b4-4bd9-a4b8-2e41a185f064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:27.523950 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6877675e-a5b4-4bd9-a4b8-2e41a185f064 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:27.524410 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6877675e-a5b4-4bd9-a4b8-2e41a185f064 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:27.525149 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6877675e-a5b4-4bd9-a4b8-2e41a185f064 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:27.526071 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6877675e-a5b4-4bd9-a4b8-2e41a185f064 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:27.528412 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6877675e-a5b4-4bd9-a4b8-2e41a185f064 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:27.541634 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 797/1604] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:36:27 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:36:27.542577 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6877675e-a5b4-4bd9-a4b8-2e41a185f064 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:27.542983 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 808/1605] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:36:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:36:27.544050 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2555cdb-74a5-4df3-92cd-427ba72995dc None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:27.557033 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2555cdb-74a5-4df3-92cd-427ba72995dc None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:27.560051 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6877675e-a5b4-4bd9-a4b8-2e41a185f064 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:27.563407 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bd1f5f-3522-4e3e-969b-878439bc9e48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:27.564801 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bd1f5f-3522-4e3e-969b-878439bc9e48 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:27.565738 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bd1f5f-3522-4e3e-969b-878439bc9e48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:27.566118 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13c5ffd-fc5c-4df1-9333-4a9074b17c9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:27.566536 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bd1f5f-3522-4e3e-969b-878439bc9e48 admin admin] PATH_INFO: `/v3/projects/9d2c9c1fb5054eeb9c45eeebda004ff2/users/59d957e121f54219b36f26ec7e023684/roles/b8c98fddc0fd4436b577fd1d3361bc38` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:27.567100 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13c5ffd-fc5c-4df1-9333-4a9074b17c9d admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:27.568524 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13c5ffd-fc5c-4df1-9333-4a9074b17c9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:27.568524 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13c5ffd-fc5c-4df1-9333-4a9074b17c9d admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:27.568766 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13c5ffd-fc5c-4df1-9333-4a9074b17c9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:27.570689 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13c5ffd-fc5c-4df1-9333-4a9074b17c9d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:27.587093 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 798/1606] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:36:27 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:36:27.588921 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13c5ffd-fc5c-4df1-9333-4a9074b17c9d admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:27.598237 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bd1f5f-3522-4e3e-969b-878439bc9e48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d2c9c1fb5054eeb9c45eeebda004ff2, user_id=59d957e121f54219b36f26ec7e023684, role_id=b8c98fddc0fd4436b577fd1d3361bc38)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:27.601590 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13c5ffd-fc5c-4df1-9333-4a9074b17c9d admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:27.604384 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bd1f5f-3522-4e3e-969b-878439bc9e48 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:27.606494 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d641c4-529f-4f71-949b-bf503fde38e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:27.607197 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d641c4-529f-4f71-949b-bf503fde38e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:27.607585 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d641c4-529f-4f71-949b-bf503fde38e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:27.607902 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d641c4-529f-4f71-949b-bf503fde38e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:27.610198 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d641c4-529f-4f71-949b-bf503fde38e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:27.616240 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d641c4-529f-4f71-949b-bf503fde38e0 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:27.636061 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 809/1607] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 15:36:27 2022] PUT /identity/v3/projects/9d2c9c1fb5054eeb9c45eeebda004ff2/users/59d957e121f54219b36f26ec7e023684/roles/b8c98fddc0fd4436b577fd1d3361bc38 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:36:27.637829 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a1f366-8ef2-428d-b261-b235a569afdd None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:27.638138 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a1f366-8ef2-428d-b261-b235a569afdd None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:27.638414 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a1f366-8ef2-428d-b261-b235a569afdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:27.662038 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 799/1608] 10.222.0.99 () {60 vars in 1152 bytes} [Thu Jul 14 15:36:27 2022] POST /identity/v3/projects => generated 371 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:36:27.663476 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d641c4-529f-4f71-949b-bf503fde38e0 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:27.722193 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-e2a1f366-8ef2-428d-b261-b235a569afdd None None] MFA Rules not processed for user `30ffdf00311048a6a47d142e5231119d`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:36:27.722193 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-e2a1f366-8ef2-428d-b261-b235a569afdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:27.860685 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d641c4-529f-4f71-949b-bf503fde38e0 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:27.866221 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f78bb0-3368-4003-8576-843c79d6c438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:27.868089 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f78bb0-3368-4003-8576-843c79d6c438 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:27.868390 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f78bb0-3368-4003-8576-843c79d6c438 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:27.868679 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f78bb0-3368-4003-8576-843c79d6c438 admin admin] PATH_INFO: `/v3/projects/636b923a8a58435484086dc8394ccc23/users/781567471d0f4ca4add4331d405febd4/roles/b8c98fddc0fd4436b577fd1d3361bc38` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:27.884707 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 810/1609] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:36:27 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:36:27.887589 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a1f366-8ef2-428d-b261-b235a569afdd None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:27.901559 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f78bb0-3368-4003-8576-843c79d6c438 admin admin] RBAC: Authorizing `identity:create_grant(project_id=636b923a8a58435484086dc8394ccc23, user_id=781567471d0f4ca4add4331d405febd4, role_id=b8c98fddc0fd4436b577fd1d3361bc38)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:27.905972 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f78bb0-3368-4003-8576-843c79d6c438 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:27.928889 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 800/1610] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 15:36:27 2022] PUT /identity/v3/projects/636b923a8a58435484086dc8394ccc23/users/781567471d0f4ca4add4331d405febd4/roles/b8c98fddc0fd4436b577fd1d3361bc38 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:36:27.929519 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f78bb0-3368-4003-8576-843c79d6c438 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:27.974244 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:27.978291 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a1f366-8ef2-428d-b261-b235a569afdd None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:28.007100 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:28.013212 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f78bb0-3368-4003-8576-843c79d6c438 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:28.019804 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f9e88b-492e-4595-8b1b-d124e943bd0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:28.020981 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f9e88b-492e-4595-8b1b-d124e943bd0f admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:28.021460 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f9e88b-492e-4595-8b1b-d124e943bd0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:28.021750 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f9e88b-492e-4595-8b1b-d124e943bd0f admin admin] PATH_INFO: `/v3/projects/67ea0d1ef31b495b8dad23f39cedf1bf/users/46f52ad195a34123982d238ea351172d/roles/b8c98fddc0fd4436b577fd1d3361bc38` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:28.043912 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f9e88b-492e-4595-8b1b-d124e943bd0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=67ea0d1ef31b495b8dad23f39cedf1bf, user_id=46f52ad195a34123982d238ea351172d, role_id=b8c98fddc0fd4436b577fd1d3361bc38)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:28.048441 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f9e88b-492e-4595-8b1b-d124e943bd0f admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:28.057453 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7de1fda-67e0-42e6-bb1a-cff72e3de544 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:28.058443 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7de1fda-67e0-42e6-bb1a-cff72e3de544 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:28.058779 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7de1fda-67e0-42e6-bb1a-cff72e3de544 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:28.059107 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7de1fda-67e0-42e6-bb1a-cff72e3de544 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:28.066643 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 801/1611] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 15:36:27 2022] PUT /identity/v3/projects/67ea0d1ef31b495b8dad23f39cedf1bf/users/46f52ad195a34123982d238ea351172d/roles/b8c98fddc0fd4436b577fd1d3361bc38 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:36:28.072246 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852656f1-5f58-4c2a-a78b-25a995f232f6 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:28.072536 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852656f1-5f58-4c2a-a78b-25a995f232f6 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:28.072983 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852656f1-5f58-4c2a-a78b-25a995f232f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:28.077876 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7de1fda-67e0-42e6-bb1a-cff72e3de544 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:28.080300 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7de1fda-67e0-42e6-bb1a-cff72e3de544 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:28.094930 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 811/1612] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:36:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:36:28.095562 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7de1fda-67e0-42e6-bb1a-cff72e3de544 service neutron] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:28.106046 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7de1fda-67e0-42e6-bb1a-cff72e3de544 service neutron] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:28.126374 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-852656f1-5f58-4c2a-a78b-25a995f232f6 None None] MFA Rules not processed for user `59d957e121f54219b36f26ec7e023684`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:36:28.127676 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-852656f1-5f58-4c2a-a78b-25a995f232f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:28.299548 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96abae2e-4126-4adb-b673-ebd26f45a4c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:28.300274 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96abae2e-4126-4adb-b673-ebd26f45a4c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:28.301779 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96abae2e-4126-4adb-b673-ebd26f45a4c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:28.302105 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96abae2e-4126-4adb-b673-ebd26f45a4c8 admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:28.302809 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96abae2e-4126-4adb-b673-ebd26f45a4c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:28.305009 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96abae2e-4126-4adb-b673-ebd26f45a4c8 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:28.391706 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 812/1613] 10.222.0.99 () {60 vars in 1143 bytes} [Thu Jul 14 15:36:28 2022] POST /identity/v3/users => generated 292 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:36:28.410237 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b00ac3-1c78-4633-8fdc-36b2e1622491 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:28.410544 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b00ac3-1c78-4633-8fdc-36b2e1622491 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:28.410836 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b00ac3-1c78-4633-8fdc-36b2e1622491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:28.412543 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 802/1614] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:36:28 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 345 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:36:28.414669 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a194e0e-2453-4177-b679-64fae86b936f None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:28.416144 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a194e0e-2453-4177-b679-64fae86b936f None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:28.416424 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a194e0e-2453-4177-b679-64fae86b936f None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:28.417963 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 803/1615] 10.222.0.99 () {54 vars in 961 bytes} [Thu Jul 14 15:36:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:36:28.418896 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad311d93-4c97-4959-8897-a82fa738c7a3 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:28.419188 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad311d93-4c97-4959-8897-a82fa738c7a3 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:28.419469 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad311d93-4c97-4959-8897-a82fa738c7a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:28.453979 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-a6b00ac3-1c78-4633-8fdc-36b2e1622491 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:36:28.455875 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-a6b00ac3-1c78-4633-8fdc-36b2e1622491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:28.458655 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-ad311d93-4c97-4959-8897-a82fa738c7a3 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:36:28.460423 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ad311d93-4c97-4959-8897-a82fa738c7a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:28.472052 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 804/1616] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:36:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:36:28.474023 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad311d93-4c97-4959-8897-a82fa738c7a3 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:28.476176 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 813/1617] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:36:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:36:28.480944 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e78c6b-bf68-4caa-ab5f-8cad137b7faf None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:28.481306 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e78c6b-bf68-4caa-ab5f-8cad137b7faf None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:28.481736 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e78c6b-bf68-4caa-ab5f-8cad137b7faf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:28.495842 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad311d93-4c97-4959-8897-a82fa738c7a3 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:28.506251 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ca85bf-14af-44c3-b2eb-6dff1ebfbd94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:28.507698 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ca85bf-14af-44c3-b2eb-6dff1ebfbd94 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:28.508420 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ca85bf-14af-44c3-b2eb-6dff1ebfbd94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:28.508747 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ca85bf-14af-44c3-b2eb-6dff1ebfbd94 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:28.510041 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ca85bf-14af-44c3-b2eb-6dff1ebfbd94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:28.524376 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ca85bf-14af-44c3-b2eb-6dff1ebfbd94 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:28.548725 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 805/1618] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:36:28 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:36:28.550609 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c2e9a9-fe6d-4dd7-835e-702f37a9b0a6 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:28.550897 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c2e9a9-fe6d-4dd7-835e-702f37a9b0a6 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:28.551182 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c2e9a9-fe6d-4dd7-835e-702f37a9b0a6 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:28.552762 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 806/1619] 10.222.0.99 () {54 vars in 961 bytes} [Thu Jul 14 15:36:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:36:28.553513 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c2e9a9-fe6d-4dd7-835e-702f37a9b0a6 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:28.554544 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-37e78c6b-bf68-4caa-ab5f-8cad137b7faf None None] MFA Rules not processed for user `3a9b1c2622934c5185f3b5f1ea623bc0`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:36:28.558126 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-37e78c6b-bf68-4caa-ab5f-8cad137b7faf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:28.679950 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c2e9a9-fe6d-4dd7-835e-702f37a9b0a6 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:28.689997 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a165e275-d7b8-4691-b05b-f0d703809dec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:28.692524 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a165e275-d7b8-4691-b05b-f0d703809dec service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:28.692828 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a165e275-d7b8-4691-b05b-f0d703809dec service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:28.693593 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a165e275-d7b8-4691-b05b-f0d703809dec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:28.728920 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a165e275-d7b8-4691-b05b-f0d703809dec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:28.732183 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a165e275-d7b8-4691-b05b-f0d703809dec service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:28.747857 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 814/1620] 10.222.0.99 () {58 vars in 1058 bytes} [Thu Jul 14 15:36:28 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:36:28.750474 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e78c6b-bf68-4caa-ab5f-8cad137b7faf None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:28.757873 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 807/1621] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:36:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:36:28.759658 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a165e275-d7b8-4691-b05b-f0d703809dec service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:28.776351 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e78c6b-bf68-4caa-ab5f-8cad137b7faf None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:28.782212 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a165e275-d7b8-4691-b05b-f0d703809dec service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:28.785350 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f954b0-895c-4390-8795-65ca6694f37c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:28.786974 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f954b0-895c-4390-8795-65ca6694f37c service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:28.787552 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f954b0-895c-4390-8795-65ca6694f37c service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:28.789063 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f954b0-895c-4390-8795-65ca6694f37c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:28.795854 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47deff6c-2db1-4591-b562-90beec6118e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:28.800423 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47deff6c-2db1-4591-b562-90beec6118e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:28.800705 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47deff6c-2db1-4591-b562-90beec6118e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:28.800992 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47deff6c-2db1-4591-b562-90beec6118e7 admin admin] PATH_INFO: `/v3/projects/67b10df1450a43c5ab006b3872288e1b/users/6e524b0e56fd48458a8733f68d50e6a5/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:28.814133 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f954b0-895c-4390-8795-65ca6694f37c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:28.817662 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f954b0-895c-4390-8795-65ca6694f37c service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:28.842908 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47deff6c-2db1-4591-b562-90beec6118e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67b10df1450a43c5ab006b3872288e1b, user_id=6e524b0e56fd48458a8733f68d50e6a5, role_id=d0f6ba1314f14d9f9c816bdb5d0c3aac)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:28.844806 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 815/1622] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:36:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:36:28.847062 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b878d5a-ee35-4074-8ac7-f29a8d0f6845 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:28.847362 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b878d5a-ee35-4074-8ac7-f29a8d0f6845 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:28.849800 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b878d5a-ee35-4074-8ac7-f29a8d0f6845 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:28.851001 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47deff6c-2db1-4591-b562-90beec6118e7 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:28.881441 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 808/1623] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 15:36:28 2022] PUT /identity/v3/projects/67b10df1450a43c5ab006b3872288e1b/users/6e524b0e56fd48458a8733f68d50e6a5/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:36:28.884248 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47deff6c-2db1-4591-b562-90beec6118e7 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:28.892824 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-3b878d5a-ee35-4074-8ac7-f29a8d0f6845 None None] MFA Rules not processed for user `3a9b1c2622934c5185f3b5f1ea623bc0`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:36:28.895244 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-3b878d5a-ee35-4074-8ac7-f29a8d0f6845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:28.903788 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:28.904254 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:29.039227 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 816/1624] 10.222.0.99 () {58 vars in 1058 bytes} [Thu Jul 14 15:36:28 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:36:29.041910 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b878d5a-ee35-4074-8ac7-f29a8d0f6845 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:29.070782 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47deff6c-2db1-4591-b562-90beec6118e7 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:29.078152 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7985da2c-823b-4865-ae37-8856a0d4cf1f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9b1c2622934c5185f3b5f1ea623bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:29.079162 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7985da2c-823b-4865-ae37-8856a0d4cf1f service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:29.079461 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7985da2c-823b-4865-ae37-8856a0d4cf1f service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:29.079774 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7985da2c-823b-4865-ae37-8856a0d4cf1f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:29.160500 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7985da2c-823b-4865-ae37-8856a0d4cf1f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:29.170171 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7985da2c-823b-4865-ae37-8856a0d4cf1f service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:29.236392 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 809/1625] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:36:28 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:36:29.236392 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7985da2c-823b-4865-ae37-8856a0d4cf1f service nova] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:29.273445 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7985da2c-823b-4865-ae37-8856a0d4cf1f service nova] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:29.293046 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a650244-cfc5-45f9-b12e-3e7663782e25 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9b1c2622934c5185f3b5f1ea623bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:29.293936 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b878d5a-ee35-4074-8ac7-f29a8d0f6845 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:29.294069 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a650244-cfc5-45f9-b12e-3e7663782e25 service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:29.294281 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a650244-cfc5-45f9-b12e-3e7663782e25 service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:29.294567 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a650244-cfc5-45f9-b12e-3e7663782e25 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:29.310510 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac490b1-3f03-4db9-a337-9b5bf4457be6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:29.311476 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac490b1-3f03-4db9-a337-9b5bf4457be6 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:29.331374 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac490b1-3f03-4db9-a337-9b5bf4457be6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:29.333525 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac490b1-3f03-4db9-a337-9b5bf4457be6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:29.336027 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac490b1-3f03-4db9-a337-9b5bf4457be6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:29.339064 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac490b1-3f03-4db9-a337-9b5bf4457be6 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:29.361610 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 817/1626] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:36:29 2022] GET /identity/v3/roles => generated 1420 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:36:29.362921 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac490b1-3f03-4db9-a337-9b5bf4457be6 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:29.392753 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac490b1-3f03-4db9-a337-9b5bf4457be6 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:29.408141 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d111fb1-84d2-4410-bf3e-e8969de6813c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:29.411257 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d111fb1-84d2-4410-bf3e-e8969de6813c admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:29.411558 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d111fb1-84d2-4410-bf3e-e8969de6813c admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:29.412481 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d111fb1-84d2-4410-bf3e-e8969de6813c admin admin] PATH_INFO: `/v3/projects/67b10df1450a43c5ab006b3872288e1b/users/6e524b0e56fd48458a8733f68d50e6a5/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:29.418549 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d111fb1-84d2-4410-bf3e-e8969de6813c admin admin] RBAC: Authorizing `identity:create_grant(project_id=67b10df1450a43c5ab006b3872288e1b, user_id=6e524b0e56fd48458a8733f68d50e6a5, role_id=d0f6ba1314f14d9f9c816bdb5d0c3aac)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:29.423661 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d111fb1-84d2-4410-bf3e-e8969de6813c admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:29.437873 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 818/1627] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 15:36:29 2022] PUT /identity/v3/projects/67b10df1450a43c5ab006b3872288e1b/users/6e524b0e56fd48458a8733f68d50e6a5/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:36:29.448605 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97de326d-7073-49f8-9619-8e38c7359e44 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:29.448902 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97de326d-7073-49f8-9619-8e38c7359e44 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:29.449518 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97de326d-7073-49f8-9619-8e38c7359e44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:29.485649 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a650244-cfc5-45f9-b12e-3e7663782e25 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:29.487574 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a650244-cfc5-45f9-b12e-3e7663782e25 service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:29.528594 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 810/1628] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:36:29 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:36:29.530177 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-97de326d-7073-49f8-9619-8e38c7359e44 None None] MFA Rules not processed for user `6e524b0e56fd48458a8733f68d50e6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:36:29.533798 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-97de326d-7073-49f8-9619-8e38c7359e44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:29.753191 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 819/1629] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:36:29 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:36:29.768795 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a650244-cfc5-45f9-b12e-3e7663782e25 service nova] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:29.906211 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:29.949674 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a650244-cfc5-45f9-b12e-3e7663782e25 service nova] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:30.037121 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04bad76-28f0-4004-b17c-45276ef50118 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:30.037662 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04bad76-28f0-4004-b17c-45276ef50118 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:30.038121 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04bad76-28f0-4004-b17c-45276ef50118 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:30.040581 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 820/1630] 10.222.0.99 () {54 vars in 912 bytes} [Thu Jul 14 15:36:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:36:30.046417 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4a68c1-12f7-4cdc-815e-e0e5fd96c35c None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:30.047431 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4a68c1-12f7-4cdc-815e-e0e5fd96c35c None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:30.047955 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4a68c1-12f7-4cdc-815e-e0e5fd96c35c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:30.064631 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b91311e-6568-40b7-90e7-49df0fb1fff1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3e1a2b83f3d4d88ba5084dfaffd22f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:30.065553 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b91311e-6568-40b7-90e7-49df0fb1fff1 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:30.065830 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b91311e-6568-40b7-90e7-49df0fb1fff1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:30.066115 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b91311e-6568-40b7-90e7-49df0fb1fff1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:30.072096 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-2c4a68c1-12f7-4cdc-815e-e0e5fd96c35c None None] MFA Rules not processed for user `3a9b1c2622934c5185f3b5f1ea623bc0`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:36:30.074108 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-2c4a68c1-12f7-4cdc-815e-e0e5fd96c35c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:30.100499 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b91311e-6568-40b7-90e7-49df0fb1fff1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:30.103910 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:30.104428 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b91311e-6568-40b7-90e7-49df0fb1fff1 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:30.136256 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 811/1631] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:36:29 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:36:30.206159 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 821/1632] 10.222.0.99 () {58 vars in 1009 bytes} [Thu Jul 14 15:36:30 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 15:36:30.220445 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b91311e-6568-40b7-90e7-49df0fb1fff1 service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:30.231127 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b91311e-6568-40b7-90e7-49df0fb1fff1 service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:30.245291 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12b01f1-23c7-4513-a7e1-b57184cecdd2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3e1a2b83f3d4d88ba5084dfaffd22f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:30.246116 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12b01f1-23c7-4513-a7e1-b57184cecdd2 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:30.246403 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12b01f1-23c7-4513-a7e1-b57184cecdd2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:30.246682 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12b01f1-23c7-4513-a7e1-b57184cecdd2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:30.271479 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12b01f1-23c7-4513-a7e1-b57184cecdd2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:30.275449 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb114b33-0d20-4f9b-a90b-a0abc115cfd0 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:30.275759 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb114b33-0d20-4f9b-a90b-a0abc115cfd0 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:30.276106 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb114b33-0d20-4f9b-a90b-a0abc115cfd0 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:30.277903 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 822/1633] 10.222.0.99 () {54 vars in 912 bytes} [Thu Jul 14 15:36:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:36:30.283450 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12b01f1-23c7-4513-a7e1-b57184cecdd2 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:30.294860 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70ae99b-bc4d-4ce6-a4af-567a82bee3a5 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:30.295059 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70ae99b-bc4d-4ce6-a4af-567a82bee3a5 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:30.295346 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70ae99b-bc4d-4ce6-a4af-567a82bee3a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:30.339058 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 812/1634] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:36:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:36:30.339963 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12b01f1-23c7-4513-a7e1-b57184cecdd2 service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:30.341398 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-d70ae99b-bc4d-4ce6-a4af-567a82bee3a5 None None] MFA Rules not processed for user `3a9b1c2622934c5185f3b5f1ea623bc0`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:36:30.342524 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-d70ae99b-bc4d-4ce6-a4af-567a82bee3a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:30.363633 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12b01f1-23c7-4513-a7e1-b57184cecdd2 service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:30.373371 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 823/1635] 10.222.0.99 () {58 vars in 1009 bytes} [Thu Jul 14 15:36:30 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:36:30.377243 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6a71c4-d5aa-4041-9cf0-6add3395fda6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3e1a2b83f3d4d88ba5084dfaffd22f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:30.377915 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6a71c4-d5aa-4041-9cf0-6add3395fda6 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:30.378183 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6a71c4-d5aa-4041-9cf0-6add3395fda6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:30.378674 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6a71c4-d5aa-4041-9cf0-6add3395fda6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:30.413908 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70ae99b-bc4d-4ce6-a4af-567a82bee3a5 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:30.436815 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70ae99b-bc4d-4ce6-a4af-567a82bee3a5 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:30.443809 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e5c37d-f85e-4985-bbac-eeb1121cd8b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3e1a2b83f3d4d88ba5084dfaffd22f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:30.444623 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e5c37d-f85e-4985-bbac-eeb1121cd8b9 service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:30.444909 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e5c37d-f85e-4985-bbac-eeb1121cd8b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:30.445190 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e5c37d-f85e-4985-bbac-eeb1121cd8b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:30.463744 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e5c37d-f85e-4985-bbac-eeb1121cd8b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:30.465449 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e5c37d-f85e-4985-bbac-eeb1121cd8b9 service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:30.488979 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 824/1636] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:36:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 15:36:30.507906 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6a71c4-d5aa-4041-9cf0-6add3395fda6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:30.509375 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6a71c4-d5aa-4041-9cf0-6add3395fda6 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:30.533405 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 813/1637] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:36:30 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:36:31.454631 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e5c37d-f85e-4985-bbac-eeb1121cd8b9 service cinder] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:31.596551 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e5c37d-f85e-4985-bbac-eeb1121cd8b9 service cinder] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:31.611109 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2f4ecc-67c6-42f1-8771-2d51aeec3cc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:31.612923 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2f4ecc-67c6-42f1-8771-2d51aeec3cc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:31.613814 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2f4ecc-67c6-42f1-8771-2d51aeec3cc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:31.614551 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2f4ecc-67c6-42f1-8771-2d51aeec3cc8 admin admin] PATH_INFO: `/v3/users/30ffdf00311048a6a47d142e5231119d` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:31.617742 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2f4ecc-67c6-42f1-8771-2d51aeec3cc8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=30ffdf00311048a6a47d142e5231119d)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:31.626226 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: WARNING py.warnings [None req-1a2f4ecc-67c6-42f1-8771-2d51aeec3cc8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 15:36:31.626226 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: warnings.warn(msg) Jul 14 15:36:31.626226 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: Jul 14 15:36:31.627823 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2f4ecc-67c6-42f1-8771-2d51aeec3cc8 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:31.785069 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-1a2f4ecc-67c6-42f1-8771-2d51aeec3cc8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30ffdf00311048a6a47d142e5231119d'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:36:31.798875 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-1a2f4ecc-67c6-42f1-8771-2d51aeec3cc8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30ffdf00311048a6a47d142e5231119d'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:36:31.847265 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-1a2f4ecc-67c6-42f1-8771-2d51aeec3cc8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30ffdf00311048a6a47d142e5231119d'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:36:31.848146 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-1a2f4ecc-67c6-42f1-8771-2d51aeec3cc8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30ffdf00311048a6a47d142e5231119d'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:36:31.848835 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-1a2f4ecc-67c6-42f1-8771-2d51aeec3cc8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30ffdf00311048a6a47d142e5231119d'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:36:31.876243 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:31.880337 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 825/1638] 10.222.0.99 () {58 vars in 1223 bytes} [Thu Jul 14 15:36:31 2022] DELETE /identity/v3/users/30ffdf00311048a6a47d142e5231119d => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:36:31.885554 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f2e080-7550-4d39-9ab9-dfcf27532002 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:31.886389 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f2e080-7550-4d39-9ab9-dfcf27532002 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:31.886807 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f2e080-7550-4d39-9ab9-dfcf27532002 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:31.919324 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-d6f2e080-7550-4d39-9ab9-dfcf27532002 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:36:31.923677 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d6f2e080-7550-4d39-9ab9-dfcf27532002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:32.119822 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 814/1639] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:36:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:36:32.161555 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2f4ecc-67c6-42f1-8771-2d51aeec3cc8 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:32.163689 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-1a2f4ecc-67c6-42f1-8771-2d51aeec3cc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:32.314549 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2f4ecc-67c6-42f1-8771-2d51aeec3cc8 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:32.324536 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5e8810-91f8-425d-be9a-f33a67f47da4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:32.325258 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5e8810-91f8-425d-be9a-f33a67f47da4 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:32.325515 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5e8810-91f8-425d-be9a-f33a67f47da4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:32.325784 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5e8810-91f8-425d-be9a-f33a67f47da4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:32.348842 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5e8810-91f8-425d-be9a-f33a67f47da4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:32.350321 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5e8810-91f8-425d-be9a-f33a67f47da4 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:32.372995 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 826/1640] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:36:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:36:34.066062 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f2e080-7550-4d39-9ab9-dfcf27532002 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:34.068947 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d6f2e080-7550-4d39-9ab9-dfcf27532002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:34.101036 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f2e080-7550-4d39-9ab9-dfcf27532002 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:34.109580 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d849c8f6-70db-49a9-a1ef-963d11d08877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:34.110951 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d849c8f6-70db-49a9-a1ef-963d11d08877 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:34.111382 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d849c8f6-70db-49a9-a1ef-963d11d08877 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:34.113104 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d849c8f6-70db-49a9-a1ef-963d11d08877 admin admin] PATH_INFO: `/v3/projects/5172d2ae5b244afd99f04b3cd6e056d0` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:34.114695 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d849c8f6-70db-49a9-a1ef-963d11d08877 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5172d2ae5b244afd99f04b3cd6e056d0)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:34.116181 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: WARNING py.warnings [None req-d849c8f6-70db-49a9-a1ef-963d11d08877 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 15:36:34.116181 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: warnings.warn(msg) Jul 14 15:36:34.116181 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: Jul 14 15:36:34.117673 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d849c8f6-70db-49a9-a1ef-963d11d08877 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:34.183413 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:34.277009 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d849c8f6-70db-49a9-a1ef-963d11d08877 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5172d2ae5b244afd99f04b3cd6e056d0', 'request_id': 'req-d849c8f6-70db-49a9-a1ef-963d11d08877'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:36:34.309782 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d849c8f6-70db-49a9-a1ef-963d11d08877 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5172d2ae5b244afd99f04b3cd6e056d0', 'request_id': 'req-d849c8f6-70db-49a9-a1ef-963d11d08877'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:36:34.335397 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d849c8f6-70db-49a9-a1ef-963d11d08877 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5172d2ae5b244afd99f04b3cd6e056d0', 'request_id': 'req-d849c8f6-70db-49a9-a1ef-963d11d08877'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:36:34.371017 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d849c8f6-70db-49a9-a1ef-963d11d08877 admin admin] The token cache is being invalidate because project 5172d2ae5b244afd99f04b3cd6e056d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 15:36:34.372350 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d849c8f6-70db-49a9-a1ef-963d11d08877 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:36:34.380535 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d849c8f6-70db-49a9-a1ef-963d11d08877 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:36:34.382494 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 815/1641] 10.222.0.99 () {58 vars in 1232 bytes} [Thu Jul 14 15:36:34 2022] DELETE /identity/v3/projects/5172d2ae5b244afd99f04b3cd6e056d0 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:36:34.389034 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab212734-4190-45ee-8f03-8ea6f015e8c9 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:34.389525 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab212734-4190-45ee-8f03-8ea6f015e8c9 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:34.389596 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab212734-4190-45ee-8f03-8ea6f015e8c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:34.461224 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-ab212734-4190-45ee-8f03-8ea6f015e8c9 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:36:34.463333 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-ab212734-4190-45ee-8f03-8ea6f015e8c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:34.689566 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d849c8f6-70db-49a9-a1ef-963d11d08877 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:34.691416 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d849c8f6-70db-49a9-a1ef-963d11d08877 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:34.762770 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 827/1642] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:36:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 369 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:36:34.764756 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2928d211-6303-479b-854c-428260a6f76e None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:34.765126 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2928d211-6303-479b-854c-428260a6f76e None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:34.765814 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2928d211-6303-479b-854c-428260a6f76e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:34.815573 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-2928d211-6303-479b-854c-428260a6f76e None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:36:34.822424 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-2928d211-6303-479b-854c-428260a6f76e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:34.852489 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 828/1643] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:36:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 15:36:34.853716 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2928d211-6303-479b-854c-428260a6f76e None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:34.879786 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2928d211-6303-479b-854c-428260a6f76e None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:34.887195 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d849c8f6-70db-49a9-a1ef-963d11d08877 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:34.889761 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0bee4d-8019-49de-b199-57acdf2c93b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:34.891749 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0bee4d-8019-49de-b199-57acdf2c93b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:34.893357 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0bee4d-8019-49de-b199-57acdf2c93b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:34.894105 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0bee4d-8019-49de-b199-57acdf2c93b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:34.896836 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0bee4d-8019-49de-b199-57acdf2c93b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:36:34.897455 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0bee4d-8019-49de-b199-57acdf2c93b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:34.901830 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0bee4d-8019-49de-b199-57acdf2c93b3 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:34.902166 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf3f4da-e4eb-4a48-ae6c-b184a4b2f600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:34.903782 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf3f4da-e4eb-4a48-ae6c-b184a4b2f600 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:34.904583 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf3f4da-e4eb-4a48-ae6c-b184a4b2f600 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:34.906382 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf3f4da-e4eb-4a48-ae6c-b184a4b2f600 admin admin] PATH_INFO: `/v3/users/59d957e121f54219b36f26ec7e023684` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:34.911268 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf3f4da-e4eb-4a48-ae6c-b184a4b2f600 admin admin] RBAC: Authorizing `identity:delete_user(user_id=59d957e121f54219b36f26ec7e023684)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:34.917413 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf3f4da-e4eb-4a48-ae6c-b184a4b2f600 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:34.938064 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 829/1644] 10.222.0.99 () {58 vars in 1152 bytes} [Thu Jul 14 15:36:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:36:34.942856 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0bee4d-8019-49de-b199-57acdf2c93b3 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:34.971654 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0bee4d-8019-49de-b199-57acdf2c93b3 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:34.995303 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07469c84-9aa4-4671-bc3c-07c1aa3c9cc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:34.996552 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07469c84-9aa4-4671-bc3c-07c1aa3c9cc2 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:34.999165 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07469c84-9aa4-4671-bc3c-07c1aa3c9cc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:35.000647 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07469c84-9aa4-4671-bc3c-07c1aa3c9cc2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:35.003465 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07469c84-9aa4-4671-bc3c-07c1aa3c9cc2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:35.008915 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07469c84-9aa4-4671-bc3c-07c1aa3c9cc2 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:35.047234 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-6cf3f4da-e4eb-4a48-ae6c-b184a4b2f600 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59d957e121f54219b36f26ec7e023684'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:36:35.059025 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-6cf3f4da-e4eb-4a48-ae6c-b184a4b2f600 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59d957e121f54219b36f26ec7e023684'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:36:35.077311 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 830/1645] 10.222.0.99 () {60 vars in 1152 bytes} [Thu Jul 14 15:36:34 2022] POST /identity/v3/projects => generated 401 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:36:35.082818 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-6cf3f4da-e4eb-4a48-ae6c-b184a4b2f600 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59d957e121f54219b36f26ec7e023684'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:36:35.084136 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07469c84-9aa4-4671-bc3c-07c1aa3c9cc2 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:35.084621 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-6cf3f4da-e4eb-4a48-ae6c-b184a4b2f600 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59d957e121f54219b36f26ec7e023684'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:36:35.088252 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-6cf3f4da-e4eb-4a48-ae6c-b184a4b2f600 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59d957e121f54219b36f26ec7e023684'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:36:35.089255 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-07469c84-9aa4-4671-bc3c-07c1aa3c9cc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:35.118166 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 816/1646] 10.222.0.99 () {58 vars in 1223 bytes} [Thu Jul 14 15:36:34 2022] DELETE /identity/v3/users/59d957e121f54219b36f26ec7e023684 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:36:35.129281 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fd6e56-b7b3-43c2-8cdc-31cdca832324 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:35.129695 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fd6e56-b7b3-43c2-8cdc-31cdca832324 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:35.130056 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fd6e56-b7b3-43c2-8cdc-31cdca832324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:35.166091 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-29fd6e56-b7b3-43c2-8cdc-31cdca832324 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:36:35.167466 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-29fd6e56-b7b3-43c2-8cdc-31cdca832324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:35.212799 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:35.220560 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07469c84-9aa4-4671-bc3c-07c1aa3c9cc2 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:35.233994 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:35.312386 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 817/1647] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:36:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:36:35.318822 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddea1e5-ff7b-4ba5-bf6b-e40dc81b3a25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:35.319739 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddea1e5-ff7b-4ba5-bf6b-e40dc81b3a25 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:35.320134 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddea1e5-ff7b-4ba5-bf6b-e40dc81b3a25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:35.320520 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddea1e5-ff7b-4ba5-bf6b-e40dc81b3a25 admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:35.322136 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddea1e5-ff7b-4ba5-bf6b-e40dc81b3a25 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:35.325894 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fd6e56-b7b3-43c2-8cdc-31cdca832324 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:35.326030 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddea1e5-ff7b-4ba5-bf6b-e40dc81b3a25 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:35.327330 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-29fd6e56-b7b3-43c2-8cdc-31cdca832324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:35.379286 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 831/1648] 10.222.0.99 () {60 vars in 1143 bytes} [Thu Jul 14 15:36:35 2022] POST /identity/v3/users => generated 301 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:36:35.380414 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddea1e5-ff7b-4ba5-bf6b-e40dc81b3a25 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:35.393736 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddea1e5-ff7b-4ba5-bf6b-e40dc81b3a25 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:35.400656 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa74ed6c-7636-4a59-b8ad-d0e09d543b7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:35.401617 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa74ed6c-7636-4a59-b8ad-d0e09d543b7d admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:35.402155 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa74ed6c-7636-4a59-b8ad-d0e09d543b7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:35.412434 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa74ed6c-7636-4a59-b8ad-d0e09d543b7d admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:35.414234 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa74ed6c-7636-4a59-b8ad-d0e09d543b7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:35.417815 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa74ed6c-7636-4a59-b8ad-d0e09d543b7d admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:35.449969 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 832/1649] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:36:35 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:36:35.453815 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faea23cd-c081-47f1-8585-f3a22ca91a6e None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:35.454285 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faea23cd-c081-47f1-8585-f3a22ca91a6e None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:35.454744 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faea23cd-c081-47f1-8585-f3a22ca91a6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:35.459644 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fd6e56-b7b3-43c2-8cdc-31cdca832324 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:35.468429 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8a4a3d-a7f1-4128-ac13-3100bcaba48d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:35.469230 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8a4a3d-a7f1-4128-ac13-3100bcaba48d service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:35.470356 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8a4a3d-a7f1-4128-ac13-3100bcaba48d service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:35.470748 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8a4a3d-a7f1-4128-ac13-3100bcaba48d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:35.484031 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8a4a3d-a7f1-4128-ac13-3100bcaba48d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:35.487637 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8a4a3d-a7f1-4128-ac13-3100bcaba48d service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:35.499271 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 818/1650] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:36:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:36:35.501106 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8a4a3d-a7f1-4128-ac13-3100bcaba48d service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:35.514717 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8a4a3d-a7f1-4128-ac13-3100bcaba48d service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:35.515850 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-faea23cd-c081-47f1-8585-f3a22ca91a6e None None] MFA Rules not processed for user `3db7ba60a24747efb55601b6e2401983`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:36:35.517387 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-faea23cd-c081-47f1-8585-f3a22ca91a6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:35.524918 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ab1156-1915-4aa6-af4a-1d46b42d263f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:35.525885 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ab1156-1915-4aa6-af4a-1d46b42d263f admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:35.526369 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ab1156-1915-4aa6-af4a-1d46b42d263f admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:35.526859 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ab1156-1915-4aa6-af4a-1d46b42d263f admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:35.527756 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ab1156-1915-4aa6-af4a-1d46b42d263f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:35.529686 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ab1156-1915-4aa6-af4a-1d46b42d263f admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:35.544344 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 819/1651] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:36:35 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:36:35.548090 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ab1156-1915-4aa6-af4a-1d46b42d263f admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:35.560748 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ab1156-1915-4aa6-af4a-1d46b42d263f admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:35.567967 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd1a5ef-8260-40b3-8d69-b7c9c964b799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:35.569237 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd1a5ef-8260-40b3-8d69-b7c9c964b799 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:35.569649 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd1a5ef-8260-40b3-8d69-b7c9c964b799 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:35.570170 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd1a5ef-8260-40b3-8d69-b7c9c964b799 admin admin] PATH_INFO: `/v3/projects/856bb3ac33614385bcf33535a36f113e/users/6ae529a7f9bd48d2afb81f653c61cf18/roles/b8c98fddc0fd4436b577fd1d3361bc38` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:35.592278 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd1a5ef-8260-40b3-8d69-b7c9c964b799 admin admin] RBAC: Authorizing `identity:create_grant(project_id=856bb3ac33614385bcf33535a36f113e, user_id=6ae529a7f9bd48d2afb81f653c61cf18, role_id=b8c98fddc0fd4436b577fd1d3361bc38)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:35.596866 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd1a5ef-8260-40b3-8d69-b7c9c964b799 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:35.631329 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 820/1652] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 15:36:35 2022] PUT /identity/v3/projects/856bb3ac33614385bcf33535a36f113e/users/6ae529a7f9bd48d2afb81f653c61cf18/roles/b8c98fddc0fd4436b577fd1d3361bc38 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 15:36:35.642332 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83273f30-039d-40cc-938e-41cf601f2b8e None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:35.642824 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83273f30-039d-40cc-938e-41cf601f2b8e None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:35.643259 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83273f30-039d-40cc-938e-41cf601f2b8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:35.666873 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 833/1653] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:36:35 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:36:35.680470 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e4382d-3657-4517-9690-11ec45bf0abf None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:35.680782 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e4382d-3657-4517-9690-11ec45bf0abf None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:35.681057 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e4382d-3657-4517-9690-11ec45bf0abf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:35.716193 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-83273f30-039d-40cc-938e-41cf601f2b8e None None] MFA Rules not processed for user `6ae529a7f9bd48d2afb81f653c61cf18`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:36:35.720735 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-83273f30-039d-40cc-938e-41cf601f2b8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:35.731353 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-e1e4382d-3657-4517-9690-11ec45bf0abf None None] MFA Rules not processed for user `3db7ba60a24747efb55601b6e2401983`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:36:35.741053 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-e1e4382d-3657-4517-9690-11ec45bf0abf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:35.949272 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 834/1655] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:36:35 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:36:35.953298 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e4382d-3657-4517-9690-11ec45bf0abf None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:35.954048 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 821/1655] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:36:35 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:36:35.963237 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83273f30-039d-40cc-938e-41cf601f2b8e None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:35.966320 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-83273f30-039d-40cc-938e-41cf601f2b8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:36.029074 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:36.062811 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:36.111241 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e4382d-3657-4517-9690-11ec45bf0abf None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:36.128238 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74f63a4-8a0f-4300-b6a9-044a176722a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:36.129124 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74f63a4-8a0f-4300-b6a9-044a176722a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:36.129492 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74f63a4-8a0f-4300-b6a9-044a176722a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:36.129868 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74f63a4-8a0f-4300-b6a9-044a176722a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:36.130656 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74f63a4-8a0f-4300-b6a9-044a176722a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:36.132752 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74f63a4-8a0f-4300-b6a9-044a176722a5 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:36.195343 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 835/1656] 10.222.0.99 () {60 vars in 1152 bytes} [Thu Jul 14 15:36:35 2022] POST /identity/v3/projects => generated 399 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:36:36.196464 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74f63a4-8a0f-4300-b6a9-044a176722a5 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:36.262984 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83273f30-039d-40cc-938e-41cf601f2b8e None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:36.346110 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74f63a4-8a0f-4300-b6a9-044a176722a5 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:36.357023 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8679203d-5b40-4f11-af74-95374b739d73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:36.357725 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8679203d-5b40-4f11-af74-95374b739d73 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:36.358015 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8679203d-5b40-4f11-af74-95374b739d73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:36.358300 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8679203d-5b40-4f11-af74-95374b739d73 admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:36.359717 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8679203d-5b40-4f11-af74-95374b739d73 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:36.361851 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8679203d-5b40-4f11-af74-95374b739d73 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:36.418255 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 836/1657] 10.222.0.99 () {60 vars in 1143 bytes} [Thu Jul 14 15:36:36 2022] POST /identity/v3/users => generated 306 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:36:36.419268 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8679203d-5b40-4f11-af74-95374b739d73 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:36.433133 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8679203d-5b40-4f11-af74-95374b739d73 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:36.449638 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9e057f-b873-49d7-b724-6061f72b33d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:36.450339 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9e057f-b873-49d7-b724-6061f72b33d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:36.450629 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9e057f-b873-49d7-b724-6061f72b33d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:36.450910 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9e057f-b873-49d7-b724-6061f72b33d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:36.451692 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9e057f-b873-49d7-b724-6061f72b33d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:36.453902 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9e057f-b873-49d7-b724-6061f72b33d7 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:36.463537 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a03d153-febd-4d3e-9268-7ad0d87b76cd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9b1c2622934c5185f3b5f1ea623bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:36.464609 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a03d153-febd-4d3e-9268-7ad0d87b76cd service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:36.465928 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a03d153-febd-4d3e-9268-7ad0d87b76cd service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:36.466895 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a03d153-febd-4d3e-9268-7ad0d87b76cd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:36.482636 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 837/1658] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:36:36 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:36:36.487083 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9e057f-b873-49d7-b724-6061f72b33d7 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:36.512406 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9e057f-b873-49d7-b724-6061f72b33d7 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:36.524583 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de262d91-0716-4e04-8721-18ab1b3a4ad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:36.525499 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de262d91-0716-4e04-8721-18ab1b3a4ad2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:36.525793 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de262d91-0716-4e04-8721-18ab1b3a4ad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:36.526074 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de262d91-0716-4e04-8721-18ab1b3a4ad2 admin admin] PATH_INFO: `/v3/projects/1616277137814fda8ac82eea4fe078e4/users/bb6e604977ea4e9aab17b0b60e68be04/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:36.572341 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de262d91-0716-4e04-8721-18ab1b3a4ad2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1616277137814fda8ac82eea4fe078e4, user_id=bb6e604977ea4e9aab17b0b60e68be04, role_id=d0f6ba1314f14d9f9c816bdb5d0c3aac)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:36.575054 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de262d91-0716-4e04-8721-18ab1b3a4ad2 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:36.605686 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a03d153-febd-4d3e-9268-7ad0d87b76cd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:36.607360 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a03d153-febd-4d3e-9268-7ad0d87b76cd service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:36.620591 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 838/1659] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 15:36:36 2022] PUT /identity/v3/projects/1616277137814fda8ac82eea4fe078e4/users/bb6e604977ea4e9aab17b0b60e68be04/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:36:36.623639 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 822/1660] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:36:35 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 660 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:36:36.624863 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de262d91-0716-4e04-8721-18ab1b3a4ad2 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:36.818441 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de262d91-0716-4e04-8721-18ab1b3a4ad2 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:36.835355 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267289af-1b2a-4118-b28b-dfb6796ac58e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:36.836142 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267289af-1b2a-4118-b28b-dfb6796ac58e admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:36.836386 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267289af-1b2a-4118-b28b-dfb6796ac58e admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:36.836671 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267289af-1b2a-4118-b28b-dfb6796ac58e admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:36.837599 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267289af-1b2a-4118-b28b-dfb6796ac58e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:36.844131 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267289af-1b2a-4118-b28b-dfb6796ac58e admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:36.848152 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:36.872153 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 839/1661] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:36:36 2022] GET /identity/v3/roles => generated 1420 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:36:36.873377 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a03d153-febd-4d3e-9268-7ad0d87b76cd service nova] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:36.899170 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a03d153-febd-4d3e-9268-7ad0d87b76cd service nova] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:36.920251 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62425dac-4426-4957-9a30-64848d6d7445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:36.923473 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62425dac-4426-4957-9a30-64848d6d7445 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:36.925324 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62425dac-4426-4957-9a30-64848d6d7445 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:36.925722 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62425dac-4426-4957-9a30-64848d6d7445 admin admin] PATH_INFO: `/v3/projects/1616277137814fda8ac82eea4fe078e4/users/bb6e604977ea4e9aab17b0b60e68be04/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:36.934952 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62425dac-4426-4957-9a30-64848d6d7445 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1616277137814fda8ac82eea4fe078e4, user_id=bb6e604977ea4e9aab17b0b60e68be04, role_id=d0f6ba1314f14d9f9c816bdb5d0c3aac)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:36.948544 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62425dac-4426-4957-9a30-64848d6d7445 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:37.001198 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 823/1662] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 15:36:36 2022] PUT /identity/v3/projects/1616277137814fda8ac82eea4fe078e4/users/bb6e604977ea4e9aab17b0b60e68be04/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:36:37.005392 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c87f6a0-c739-4eca-9e21-36cb90602013 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:37.005694 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c87f6a0-c739-4eca-9e21-36cb90602013 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:37.005984 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c87f6a0-c739-4eca-9e21-36cb90602013 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:37.075918 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-0c87f6a0-c739-4eca-9e21-36cb90602013 None None] MFA Rules not processed for user `bb6e604977ea4e9aab17b0b60e68be04`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:36:37.077264 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-0c87f6a0-c739-4eca-9e21-36cb90602013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:37.283275 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 840/1663] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:36:37 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:36:37.313005 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62425dac-4426-4957-9a30-64848d6d7445 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:37.315911 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-62425dac-4426-4957-9a30-64848d6d7445 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:37.470372 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:37.513900 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62425dac-4426-4957-9a30-64848d6d7445 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:37.523715 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfb4201-fdce-49fc-81ef-ef8e84edcf34 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3e1a2b83f3d4d88ba5084dfaffd22f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:37.524771 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfb4201-fdce-49fc-81ef-ef8e84edcf34 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:37.524914 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfb4201-fdce-49fc-81ef-ef8e84edcf34 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:37.525147 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfb4201-fdce-49fc-81ef-ef8e84edcf34 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:37.541998 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfb4201-fdce-49fc-81ef-ef8e84edcf34 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:37.544634 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfb4201-fdce-49fc-81ef-ef8e84edcf34 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:37.587178 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 824/1664] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:36:37 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:36:37.701704 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c87f6a0-c739-4eca-9e21-36cb90602013 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:37.719073 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c87f6a0-c739-4eca-9e21-36cb90602013 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:37.741113 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfa41a9-277b-418a-862a-032950a034a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3e1a2b83f3d4d88ba5084dfaffd22f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:37.741784 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfa41a9-277b-418a-862a-032950a034a4 service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:37.742076 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfa41a9-277b-418a-862a-032950a034a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:37.742355 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfa41a9-277b-418a-862a-032950a034a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:37.919029 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfa41a9-277b-418a-862a-032950a034a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:37.923519 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfa41a9-277b-418a-862a-032950a034a4 service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:37.946277 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 841/1665] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:36:37 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:36:37.953402 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfb4201-fdce-49fc-81ef-ef8e84edcf34 service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:37.955727 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-7cfb4201-fdce-49fc-81ef-ef8e84edcf34 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:38.153527 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfb4201-fdce-49fc-81ef-ef8e84edcf34 service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:38.167812 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45d0513-32f4-4573-ae75-4535d0cff97f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:38.168790 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45d0513-32f4-4573-ae75-4535d0cff97f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:38.173748 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45d0513-32f4-4573-ae75-4535d0cff97f admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:38.174130 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45d0513-32f4-4573-ae75-4535d0cff97f admin admin] PATH_INFO: `/v3/projects/9d2c9c1fb5054eeb9c45eeebda004ff2` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:38.175628 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45d0513-32f4-4573-ae75-4535d0cff97f admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d2c9c1fb5054eeb9c45eeebda004ff2)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:38.180244 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45d0513-32f4-4573-ae75-4535d0cff97f admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:38.278226 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a45d0513-32f4-4573-ae75-4535d0cff97f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d2c9c1fb5054eeb9c45eeebda004ff2', 'request_id': 'req-a45d0513-32f4-4573-ae75-4535d0cff97f'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:36:38.299297 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a45d0513-32f4-4573-ae75-4535d0cff97f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d2c9c1fb5054eeb9c45eeebda004ff2', 'request_id': 'req-a45d0513-32f4-4573-ae75-4535d0cff97f'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:36:38.317401 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a45d0513-32f4-4573-ae75-4535d0cff97f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d2c9c1fb5054eeb9c45eeebda004ff2', 'request_id': 'req-a45d0513-32f4-4573-ae75-4535d0cff97f'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:36:38.337713 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b7a7b0-35f7-47d9-a1a3-134b37d15498 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:38.338097 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b7a7b0-35f7-47d9-a1a3-134b37d15498 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:38.338917 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b7a7b0-35f7-47d9-a1a3-134b37d15498 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:38.340806 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 842/1666] 10.222.0.99 () {54 vars in 912 bytes} [Thu Jul 14 15:36:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 15:36:38.352115 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc76f8c-e0ae-4812-aeee-ccaf12232fb3 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:38.352533 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc76f8c-e0ae-4812-aeee-ccaf12232fb3 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:38.352936 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc76f8c-e0ae-4812-aeee-ccaf12232fb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:38.385623 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a45d0513-32f4-4573-ae75-4535d0cff97f admin admin] The token cache is being invalidate because project 9d2c9c1fb5054eeb9c45eeebda004ff2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 15:36:38.387019 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a45d0513-32f4-4573-ae75-4535d0cff97f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:36:38.389884 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a45d0513-32f4-4573-ae75-4535d0cff97f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:36:38.403055 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 825/1667] 10.222.0.99 () {58 vars in 1232 bytes} [Thu Jul 14 15:36:37 2022] DELETE /identity/v3/projects/9d2c9c1fb5054eeb9c45eeebda004ff2 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:36:38.408171 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47cbd80-c1c6-4b8c-a78b-c4486e43d7fe None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:38.408496 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47cbd80-c1c6-4b8c-a78b-c4486e43d7fe None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:38.408786 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47cbd80-c1c6-4b8c-a78b-c4486e43d7fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:38.442639 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-a47cbd80-c1c6-4b8c-a78b-c4486e43d7fe None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:36:38.444736 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a47cbd80-c1c6-4b8c-a78b-c4486e43d7fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:38.511552 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-efc76f8c-e0ae-4812-aeee-ccaf12232fb3 None None] MFA Rules not processed for user `0d0261759f9743b1a13a760ca9159305`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:36:38.512887 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-efc76f8c-e0ae-4812-aeee-ccaf12232fb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:38.660914 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 826/1668] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:36:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:36:38.675003 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f88e77c-eca7-4357-a59a-f5e788350fe4 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:38.675357 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f88e77c-eca7-4357-a59a-f5e788350fe4 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:38.675669 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f88e77c-eca7-4357-a59a-f5e788350fe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:38.692100 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 843/1669] 10.222.0.99 () {58 vars in 1009 bytes} [Thu Jul 14 15:36:38 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 341 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 15:36:38.725274 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-0f88e77c-eca7-4357-a59a-f5e788350fe4 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:36:38.726542 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0f88e77c-eca7-4357-a59a-f5e788350fe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:38.771818 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc76f8c-e0ae-4812-aeee-ccaf12232fb3 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:38.773916 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-efc76f8c-e0ae-4812-aeee-ccaf12232fb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:38.778063 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 827/1670] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:36:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:36:38.790803 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f88e77c-eca7-4357-a59a-f5e788350fe4 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:38.801483 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc76f8c-e0ae-4812-aeee-ccaf12232fb3 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:38.814433 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f88e77c-eca7-4357-a59a-f5e788350fe4 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:38.820182 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11e21b6-7b8c-4182-8751-6aac2420c7ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:38.821232 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11e21b6-7b8c-4182-8751-6aac2420c7ed admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:38.821712 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11e21b6-7b8c-4182-8751-6aac2420c7ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:38.822148 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11e21b6-7b8c-4182-8751-6aac2420c7ed admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:38.823301 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11e21b6-7b8c-4182-8751-6aac2420c7ed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:36:38.823868 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11e21b6-7b8c-4182-8751-6aac2420c7ed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:38.829793 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f5d9a2-9c04-4c69-9a09-ace94a7245a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:38.830490 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f5d9a2-9c04-4c69-9a09-ace94a7245a5 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:38.830782 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f5d9a2-9c04-4c69-9a09-ace94a7245a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:38.832908 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f5d9a2-9c04-4c69-9a09-ace94a7245a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:38.838575 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11e21b6-7b8c-4182-8751-6aac2420c7ed admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:38.848941 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:38.852103 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f5d9a2-9c04-4c69-9a09-ace94a7245a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:38.853806 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f5d9a2-9c04-4c69-9a09-ace94a7245a5 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:38.859973 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 828/1671] 10.222.0.99 () {58 vars in 1152 bytes} [Thu Jul 14 15:36:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:36:38.864397 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11e21b6-7b8c-4182-8751-6aac2420c7ed admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:38.872469 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 844/1672] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:36:38 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 15:36:38.896449 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11e21b6-7b8c-4182-8751-6aac2420c7ed admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:38.908473 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8342594-4db8-400d-b4b0-a3eb89c8212a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:38.909208 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8342594-4db8-400d-b4b0-a3eb89c8212a admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:38.909446 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8342594-4db8-400d-b4b0-a3eb89c8212a admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:38.909733 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8342594-4db8-400d-b4b0-a3eb89c8212a admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:38.910444 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8342594-4db8-400d-b4b0-a3eb89c8212a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:38.917799 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8342594-4db8-400d-b4b0-a3eb89c8212a admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:38.972817 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 829/1673] 10.222.0.99 () {60 vars in 1152 bytes} [Thu Jul 14 15:36:38 2022] POST /identity/v3/projects => generated 405 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:36:38.973947 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f5d9a2-9c04-4c69-9a09-ace94a7245a5 service neutron] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:38.982880 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:39.006016 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f5d9a2-9c04-4c69-9a09-ace94a7245a5 service neutron] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:39.197800 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e4327b-d6a6-4ca9-b301-19a01c76b5ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:39.198507 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e4327b-d6a6-4ca9-b301-19a01c76b5ea admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:39.198792 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e4327b-d6a6-4ca9-b301-19a01c76b5ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:39.199067 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e4327b-d6a6-4ca9-b301-19a01c76b5ea admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:39.199814 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e4327b-d6a6-4ca9-b301-19a01c76b5ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:39.201797 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e4327b-d6a6-4ca9-b301-19a01c76b5ea admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:39.246656 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 845/1674] 10.222.0.99 () {60 vars in 1143 bytes} [Thu Jul 14 15:36:38 2022] POST /identity/v3/users => generated 303 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:36:39.251215 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8342594-4db8-400d-b4b0-a3eb89c8212a admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:39.267568 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8342594-4db8-400d-b4b0-a3eb89c8212a admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:39.285458 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8542d3-9729-45ce-b959-e37727f06665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:39.286465 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8542d3-9729-45ce-b959-e37727f06665 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:39.286465 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8542d3-9729-45ce-b959-e37727f06665 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:39.286793 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8542d3-9729-45ce-b959-e37727f06665 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:39.288197 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8542d3-9729-45ce-b959-e37727f06665 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:39.289907 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8542d3-9729-45ce-b959-e37727f06665 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:39.314936 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 830/1675] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:36:39 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:36:39.319712 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e4327b-d6a6-4ca9-b301-19a01c76b5ea admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:39.336750 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e4327b-d6a6-4ca9-b301-19a01c76b5ea admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:39.354361 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a889261-8895-4b1c-874f-cf88be972fa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:39.355228 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a889261-8895-4b1c-874f-cf88be972fa9 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:39.356581 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a889261-8895-4b1c-874f-cf88be972fa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:39.356879 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a889261-8895-4b1c-874f-cf88be972fa9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:39.360277 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a889261-8895-4b1c-874f-cf88be972fa9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:39.366294 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a889261-8895-4b1c-874f-cf88be972fa9 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:39.393457 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 846/1676] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:36:39 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:36:39.397799 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8542d3-9729-45ce-b959-e37727f06665 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:39.433320 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8542d3-9729-45ce-b959-e37727f06665 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:39.443080 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf724263-8922-4242-89f6-6a9cfe93cf47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:39.444034 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf724263-8922-4242-89f6-6a9cfe93cf47 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:39.444330 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf724263-8922-4242-89f6-6a9cfe93cf47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:39.444615 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf724263-8922-4242-89f6-6a9cfe93cf47 admin admin] PATH_INFO: `/v3/projects/0d3d704462bc4886b997f04ba885de7a/users/1c43928aa8cf4d729873ae9361dcb532/roles/b8c98fddc0fd4436b577fd1d3361bc38` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:39.467355 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf724263-8922-4242-89f6-6a9cfe93cf47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d3d704462bc4886b997f04ba885de7a, user_id=1c43928aa8cf4d729873ae9361dcb532, role_id=b8c98fddc0fd4436b577fd1d3361bc38)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:39.471093 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf724263-8922-4242-89f6-6a9cfe93cf47 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:39.492527 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 831/1677] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 15:36:39 2022] PUT /identity/v3/projects/0d3d704462bc4886b997f04ba885de7a/users/1c43928aa8cf4d729873ae9361dcb532/roles/b8c98fddc0fd4436b577fd1d3361bc38 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 15:36:39.497523 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dcfc58-8f40-4bd3-ac5c-25948b3f4cf2 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:39.497981 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dcfc58-8f40-4bd3-ac5c-25948b3f4cf2 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:39.498261 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dcfc58-8f40-4bd3-ac5c-25948b3f4cf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:39.526319 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-a7dcfc58-8f40-4bd3-ac5c-25948b3f4cf2 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:36:39.527655 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-a7dcfc58-8f40-4bd3-ac5c-25948b3f4cf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:39.668229 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 847/1678] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:36:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:36:39.695430 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf724263-8922-4242-89f6-6a9cfe93cf47 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:39.698078 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-bf724263-8922-4242-89f6-6a9cfe93cf47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:39.849796 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf724263-8922-4242-89f6-6a9cfe93cf47 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:39.859796 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48957525-f94c-4c52-8326-06466d9379ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:39.860477 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48957525-f94c-4c52-8326-06466d9379ab service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:39.860773 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48957525-f94c-4c52-8326-06466d9379ab service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:39.861075 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48957525-f94c-4c52-8326-06466d9379ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:39.888645 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48957525-f94c-4c52-8326-06466d9379ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:39.891192 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48957525-f94c-4c52-8326-06466d9379ab service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:39.906743 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 832/1679] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:36:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:36:40.670732 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668dcc4d-62ed-427d-bdd4-3cc03b1f65ee None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:40.671472 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668dcc4d-62ed-427d-bdd4-3cc03b1f65ee None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:40.671472 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668dcc4d-62ed-427d-bdd4-3cc03b1f65ee None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:40.674401 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 848/1680] 10.222.0.99 () {54 vars in 960 bytes} [Thu Jul 14 15:36:40 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:36:40.689047 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5bf5c4-b63a-4cf2-8a76-44a4f3e1cf4a None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:40.690010 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5bf5c4-b63a-4cf2-8a76-44a4f3e1cf4a None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:40.690448 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5bf5c4-b63a-4cf2-8a76-44a4f3e1cf4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:40.767153 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-0d5bf5c4-b63a-4cf2-8a76-44a4f3e1cf4a None None] MFA Rules not processed for user `dc157c485f794e968a60d8405c9310ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:36:40.775123 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0d5bf5c4-b63a-4cf2-8a76-44a4f3e1cf4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:41.049430 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 833/1681] 10.222.0.99 () {58 vars in 1057 bytes} [Thu Jul 14 15:36:40 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 367 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 15:36:41.057418 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668dcc4d-62ed-427d-bdd4-3cc03b1f65ee None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:41.085275 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668dcc4d-62ed-427d-bdd4-3cc03b1f65ee None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:41.104356 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cca131b-0a8f-47e9-bb9d-ba95c7440344 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc157c485f794e968a60d8405c9310ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:41.105040 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cca131b-0a8f-47e9-bb9d-ba95c7440344 service placement] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:41.105329 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cca131b-0a8f-47e9-bb9d-ba95c7440344 service placement] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:41.105617 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cca131b-0a8f-47e9-bb9d-ba95c7440344 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:41.126115 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-2cca131b-0a8f-47e9-bb9d-ba95c7440344 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:41.155410 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cca131b-0a8f-47e9-bb9d-ba95c7440344 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:41.157520 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cca131b-0a8f-47e9-bb9d-ba95c7440344 service placement] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:41.195982 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 849/1682] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:36:41 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:36:42.721236 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e803366d-d19b-472e-b13b-419d52b7c561 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:42.723179 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e803366d-d19b-472e-b13b-419d52b7c561 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:42.723476 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e803366d-d19b-472e-b13b-419d52b7c561 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:42.830572 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-e803366d-d19b-472e-b13b-419d52b7c561 None None] MFA Rules not processed for user `781567471d0f4ca4add4331d405febd4`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:36:42.835628 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e803366d-d19b-472e-b13b-419d52b7c561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:42.930484 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:43.078818 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 834/1683] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:36:42 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 367 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 15:36:43.094587 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cca131b-0a8f-47e9-bb9d-ba95c7440344 service placement] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:43.102496 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-2cca131b-0a8f-47e9-bb9d-ba95c7440344 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:43.397733 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9192892-5366-47b2-9ccc-903bbd170c19 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:43.398030 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9192892-5366-47b2-9ccc-903bbd170c19 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:43.398317 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9192892-5366-47b2-9ccc-903bbd170c19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:43.432833 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-a9192892-5366-47b2-9ccc-903bbd170c19 None None] MFA Rules not processed for user `1c43928aa8cf4d729873ae9361dcb532`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:36:43.434827 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a9192892-5366-47b2-9ccc-903bbd170c19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:43.459068 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cca131b-0a8f-47e9-bb9d-ba95c7440344 service placement] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:43.476514 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c28cfcf-236a-47d8-99e3-3fface1bd45f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9b1c2622934c5185f3b5f1ea623bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:43.479372 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c28cfcf-236a-47d8-99e3-3fface1bd45f service nova] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:43.479960 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c28cfcf-236a-47d8-99e3-3fface1bd45f service nova] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:43.481317 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c28cfcf-236a-47d8-99e3-3fface1bd45f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:43.497760 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:43.512779 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c28cfcf-236a-47d8-99e3-3fface1bd45f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:43.516807 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c28cfcf-236a-47d8-99e3-3fface1bd45f service nova] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:43.538441 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 850/1684] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:36:43 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:36:43.582014 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c28cfcf-236a-47d8-99e3-3fface1bd45f service nova] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:43.583456 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-6c28cfcf-236a-47d8-99e3-3fface1bd45f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:43.618218 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 835/1685] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:36:43 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 15:36:43.619201 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9192892-5366-47b2-9ccc-903bbd170c19 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:43.652491 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c28cfcf-236a-47d8-99e3-3fface1bd45f service nova] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:43.665689 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe0de88-5686-4bc2-a576-db8b6bfd614e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:43.673570 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe0de88-5686-4bc2-a576-db8b6bfd614e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:43.673875 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe0de88-5686-4bc2-a576-db8b6bfd614e admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:43.674168 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe0de88-5686-4bc2-a576-db8b6bfd614e admin admin] PATH_INFO: `/v3/users/b27ce02ec0f5452d80820cf92c4e77ba` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:43.677376 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe0de88-5686-4bc2-a576-db8b6bfd614e admin admin] RBAC: Authorizing `identity:delete_user(user_id=b27ce02ec0f5452d80820cf92c4e77ba)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:43.681685 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe0de88-5686-4bc2-a576-db8b6bfd614e admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:43.685188 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9192892-5366-47b2-9ccc-903bbd170c19 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:43.704616 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31c4a24-2888-44f8-983b-9b2064e7c655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:43.708709 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31c4a24-2888-44f8-983b-9b2064e7c655 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:43.710122 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31c4a24-2888-44f8-983b-9b2064e7c655 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:43.710423 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31c4a24-2888-44f8-983b-9b2064e7c655 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:43.714835 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31c4a24-2888-44f8-983b-9b2064e7c655 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:43.720573 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31c4a24-2888-44f8-983b-9b2064e7c655 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:43.811214 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 836/1686] 10.222.0.99 () {60 vars in 1152 bytes} [Thu Jul 14 15:36:43 2022] POST /identity/v3/projects => generated 403 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:36:43.818132 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4de7bf1-8336-4171-8aa3-699aed6e5676 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:43.818978 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4de7bf1-8336-4171-8aa3-699aed6e5676 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:43.819436 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4de7bf1-8336-4171-8aa3-699aed6e5676 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:43.829712 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-5fe0de88-5686-4bc2-a576-db8b6bfd614e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b27ce02ec0f5452d80820cf92c4e77ba'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:36:43.842108 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-5fe0de88-5686-4bc2-a576-db8b6bfd614e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b27ce02ec0f5452d80820cf92c4e77ba'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:36:43.866350 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-5fe0de88-5686-4bc2-a576-db8b6bfd614e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b27ce02ec0f5452d80820cf92c4e77ba'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:36:43.867727 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-5fe0de88-5686-4bc2-a576-db8b6bfd614e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b27ce02ec0f5452d80820cf92c4e77ba'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:36:43.868692 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-5fe0de88-5686-4bc2-a576-db8b6bfd614e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b27ce02ec0f5452d80820cf92c4e77ba'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:36:43.902037 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-b4de7bf1-8336-4171-8aa3-699aed6e5676 None None] MFA Rules not processed for user `d01c301ea20d4acab8dff1148501a975`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:36:43.903306 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b4de7bf1-8336-4171-8aa3-699aed6e5676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:43.927148 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 851/1687] 10.222.0.99 () {58 vars in 1223 bytes} [Thu Jul 14 15:36:43 2022] DELETE /identity/v3/users/b27ce02ec0f5452d80820cf92c4e77ba => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 15:36:43.942063 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe0de88-5686-4bc2-a576-db8b6bfd614e admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:43.946057 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-5fe0de88-5686-4bc2-a576-db8b6bfd614e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:44.078483 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 837/1688] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:36:43 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:36:44.080610 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf28c69-5c30-4086-8171-b5aabc1466a5 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:44.080949 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf28c69-5c30-4086-8171-b5aabc1466a5 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:44.081276 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf28c69-5c30-4086-8171-b5aabc1466a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:44.120388 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-9bf28c69-5c30-4086-8171-b5aabc1466a5 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:36:44.122920 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-9bf28c69-5c30-4086-8171-b5aabc1466a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:44.169755 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:44.222016 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe0de88-5686-4bc2-a576-db8b6bfd614e admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:44.242538 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904f459f-3f40-4532-8ef2-54e552a9e5ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:44.243991 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904f459f-3f40-4532-8ef2-54e552a9e5ae admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:44.245682 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904f459f-3f40-4532-8ef2-54e552a9e5ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:44.246206 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904f459f-3f40-4532-8ef2-54e552a9e5ae admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:44.247204 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904f459f-3f40-4532-8ef2-54e552a9e5ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:44.253490 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904f459f-3f40-4532-8ef2-54e552a9e5ae admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:44.303733 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 838/1689] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:36:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:36:44.305484 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d626d9b-0e63-4ef4-9b1b-ac2b0add3398 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:44.308253 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d626d9b-0e63-4ef4-9b1b-ac2b0add3398 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:44.308551 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d626d9b-0e63-4ef4-9b1b-ac2b0add3398 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:44.323887 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 852/1690] 10.222.0.99 () {60 vars in 1143 bytes} [Thu Jul 14 15:36:43 2022] POST /identity/v3/users => generated 308 bytes in 395 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:36:44.326213 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904f459f-3f40-4532-8ef2-54e552a9e5ae admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:44.360383 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904f459f-3f40-4532-8ef2-54e552a9e5ae admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:44.365318 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-0d626d9b-0e63-4ef4-9b1b-ac2b0add3398 None None] MFA Rules not processed for user `d01c301ea20d4acab8dff1148501a975`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:36:44.366922 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0d626d9b-0e63-4ef4-9b1b-ac2b0add3398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:44.378451 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a28f4a4-19b6-4bf6-8f06-30d0c19a24b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:44.379337 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a28f4a4-19b6-4bf6-8f06-30d0c19a24b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:44.379896 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a28f4a4-19b6-4bf6-8f06-30d0c19a24b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:44.380388 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a28f4a4-19b6-4bf6-8f06-30d0c19a24b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:44.381400 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a28f4a4-19b6-4bf6-8f06-30d0c19a24b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:44.386750 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a28f4a4-19b6-4bf6-8f06-30d0c19a24b3 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:44.420213 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 853/1691] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:36:44 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:36:44.421357 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a28f4a4-19b6-4bf6-8f06-30d0c19a24b3 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:44.424276 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-7a28f4a4-19b6-4bf6-8f06-30d0c19a24b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:44.563992 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:44.573280 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 839/1692] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:36:44 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:36:44.575526 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d626d9b-0e63-4ef4-9b1b-ac2b0add3398 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:44.581092 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a28f4a4-19b6-4bf6-8f06-30d0c19a24b3 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:44.590133 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d626d9b-0e63-4ef4-9b1b-ac2b0add3398 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:44.594153 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730ae00c-f7d1-4ed2-9854-52277c3e5527 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:44.595099 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730ae00c-f7d1-4ed2-9854-52277c3e5527 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:44.597513 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730ae00c-f7d1-4ed2-9854-52277c3e5527 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:44.598510 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730ae00c-f7d1-4ed2-9854-52277c3e5527 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:44.608965 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b357faf4-dc90-4d5c-b359-88615a3c5daa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:44.612998 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b357faf4-dc90-4d5c-b359-88615a3c5daa admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:44.613482 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b357faf4-dc90-4d5c-b359-88615a3c5daa admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:44.613815 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b357faf4-dc90-4d5c-b359-88615a3c5daa admin admin] PATH_INFO: `/v3/projects/599311d17a5e45ecb73ebeaff4393da9/users/7b8e301214be4ffd8c7d39bbddf36f34/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:44.628363 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730ae00c-f7d1-4ed2-9854-52277c3e5527 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:44.630100 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730ae00c-f7d1-4ed2-9854-52277c3e5527 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:44.643878 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 854/1693] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:36:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:36:44.644861 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730ae00c-f7d1-4ed2-9854-52277c3e5527 service neutron] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:44.646683 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-730ae00c-f7d1-4ed2-9854-52277c3e5527 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:44.662502 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b357faf4-dc90-4d5c-b359-88615a3c5daa admin admin] RBAC: Authorizing `identity:create_grant(project_id=599311d17a5e45ecb73ebeaff4393da9, user_id=7b8e301214be4ffd8c7d39bbddf36f34, role_id=d0f6ba1314f14d9f9c816bdb5d0c3aac)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:44.675791 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b357faf4-dc90-4d5c-b359-88615a3c5daa admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:44.714313 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 840/1694] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 15:36:44 2022] PUT /identity/v3/projects/599311d17a5e45ecb73ebeaff4393da9/users/7b8e301214be4ffd8c7d39bbddf36f34/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:36:44.716089 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b357faf4-dc90-4d5c-b359-88615a3c5daa admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:44.821643 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730ae00c-f7d1-4ed2-9854-52277c3e5527 service neutron] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:44.914592 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b357faf4-dc90-4d5c-b359-88615a3c5daa admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:44.928308 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b187d3e1-89cd-46f6-b403-823f454eb096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:44.929290 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b187d3e1-89cd-46f6-b403-823f454eb096 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:44.929949 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b187d3e1-89cd-46f6-b403-823f454eb096 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:44.930436 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b187d3e1-89cd-46f6-b403-823f454eb096 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:44.931398 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b187d3e1-89cd-46f6-b403-823f454eb096 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:44.934783 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b187d3e1-89cd-46f6-b403-823f454eb096 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:44.950948 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 841/1695] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:36:44 2022] GET /identity/v3/roles => generated 1420 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:36:44.955633 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b187d3e1-89cd-46f6-b403-823f454eb096 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:44.963334 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83f2c1c-4391-4015-b6d7-45c01bc8922f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9b1c2622934c5185f3b5f1ea623bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:44.964281 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83f2c1c-4391-4015-b6d7-45c01bc8922f service nova] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:44.964824 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83f2c1c-4391-4015-b6d7-45c01bc8922f service nova] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:44.965235 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83f2c1c-4391-4015-b6d7-45c01bc8922f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:45.011709 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b187d3e1-89cd-46f6-b403-823f454eb096 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:45.035319 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bdb546-64b9-4a7e-aa26-6aaa04407ba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:45.052014 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bdb546-64b9-4a7e-aa26-6aaa04407ba8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:45.052832 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bdb546-64b9-4a7e-aa26-6aaa04407ba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:45.053316 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bdb546-64b9-4a7e-aa26-6aaa04407ba8 admin admin] PATH_INFO: `/v3/projects/599311d17a5e45ecb73ebeaff4393da9/users/7b8e301214be4ffd8c7d39bbddf36f34/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:45.058681 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bdb546-64b9-4a7e-aa26-6aaa04407ba8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=599311d17a5e45ecb73ebeaff4393da9, user_id=7b8e301214be4ffd8c7d39bbddf36f34, role_id=d0f6ba1314f14d9f9c816bdb5d0c3aac)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:45.068405 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bdb546-64b9-4a7e-aa26-6aaa04407ba8 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:45.090461 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83f2c1c-4391-4015-b6d7-45c01bc8922f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:45.095987 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83f2c1c-4391-4015-b6d7-45c01bc8922f service nova] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:45.117541 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 842/1696] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 15:36:44 2022] PUT /identity/v3/projects/599311d17a5e45ecb73ebeaff4393da9/users/7b8e301214be4ffd8c7d39bbddf36f34/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 15:36:45.132042 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131eb8c5-d0c4-4699-90ef-e2deed152f01 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:45.132334 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131eb8c5-d0c4-4699-90ef-e2deed152f01 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:45.137321 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131eb8c5-d0c4-4699-90ef-e2deed152f01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:45.151497 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 855/1697] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:36:44 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 507 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:36:45.229842 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:45.234606 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-131eb8c5-d0c4-4699-90ef-e2deed152f01 None None] MFA Rules not processed for user `adaeadf613954d7393b3fd3bae68c362`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:36:45.236817 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-131eb8c5-d0c4-4699-90ef-e2deed152f01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:45.409580 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 843/1698] 10.222.0.99 () {58 vars in 1009 bytes} [Thu Jul 14 15:36:45 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 15:36:45.592545 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83f2c1c-4391-4015-b6d7-45c01bc8922f service nova] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:45.594428 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-a83f2c1c-4391-4015-b6d7-45c01bc8922f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:45.613357 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83f2c1c-4391-4015-b6d7-45c01bc8922f service nova] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:45.647187 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131eb8c5-d0c4-4699-90ef-e2deed152f01 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:45.650891 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-131eb8c5-d0c4-4699-90ef-e2deed152f01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:45.698463 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:45.792891 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c8887c-485d-45ce-b2c7-e01c3361d743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:45.793724 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c8887c-485d-45ce-b2c7-e01c3361d743 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:45.794017 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c8887c-485d-45ce-b2c7-e01c3361d743 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:45.794301 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c8887c-485d-45ce-b2c7-e01c3361d743 admin admin] PATH_INFO: `/v3/users/6ae529a7f9bd48d2afb81f653c61cf18` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:45.797338 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c8887c-485d-45ce-b2c7-e01c3361d743 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ae529a7f9bd48d2afb81f653c61cf18)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:45.799324 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c8887c-485d-45ce-b2c7-e01c3361d743 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:45.850102 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131eb8c5-d0c4-4699-90ef-e2deed152f01 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:45.860393 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218d61b5-3b83-45fd-847d-ca2af64eb742 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd2ec3b3ebc4aacb322c95270374451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:45.863106 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218d61b5-3b83-45fd-847d-ca2af64eb742 service swift] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:45.863106 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218d61b5-3b83-45fd-847d-ca2af64eb742 service swift] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:45.863464 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218d61b5-3b83-45fd-847d-ca2af64eb742 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:45.898302 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218d61b5-3b83-45fd-847d-ca2af64eb742 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:45.903008 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218d61b5-3b83-45fd-847d-ca2af64eb742 service swift] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:45.914339 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 844/1699] 10.222.0.99 () {60 vars in 1384 bytes} [Thu Jul 14 15:36:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 15:36:45.942735 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-f6c8887c-485d-45ce-b2c7-e01c3361d743 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ae529a7f9bd48d2afb81f653c61cf18'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:36:45.956146 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-f6c8887c-485d-45ce-b2c7-e01c3361d743 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ae529a7f9bd48d2afb81f653c61cf18'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:36:46.002650 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-f6c8887c-485d-45ce-b2c7-e01c3361d743 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ae529a7f9bd48d2afb81f653c61cf18'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:36:46.007035 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-f6c8887c-485d-45ce-b2c7-e01c3361d743 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ae529a7f9bd48d2afb81f653c61cf18'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:36:46.008026 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-f6c8887c-485d-45ce-b2c7-e01c3361d743 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ae529a7f9bd48d2afb81f653c61cf18'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:36:46.043268 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 856/1700] 10.222.0.99 () {58 vars in 1223 bytes} [Thu Jul 14 15:36:45 2022] DELETE /identity/v3/users/6ae529a7f9bd48d2afb81f653c61cf18 => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:36:46.047756 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a641fbb0-577e-47f0-807d-f2c8bb864513 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:46.048744 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a641fbb0-577e-47f0-807d-f2c8bb864513 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:46.049283 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a641fbb0-577e-47f0-807d-f2c8bb864513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:46.102209 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-a641fbb0-577e-47f0-807d-f2c8bb864513 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:36:46.111941 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a641fbb0-577e-47f0-807d-f2c8bb864513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:46.367292 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 845/1701] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:36:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:36:46.409568 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c8887c-485d-45ce-b2c7-e01c3361d743 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:46.411240 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-f6c8887c-485d-45ce-b2c7-e01c3361d743 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:46.585193 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c8887c-485d-45ce-b2c7-e01c3361d743 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:46.591264 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7028ca15-7531-4e37-ba06-60791f815b4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:46.592195 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7028ca15-7531-4e37-ba06-60791f815b4a service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:46.593869 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7028ca15-7531-4e37-ba06-60791f815b4a service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:46.594912 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7028ca15-7531-4e37-ba06-60791f815b4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:46.619378 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428bb36b-b70a-4a2b-a5a1-1222b9cb1c4c None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:46.619572 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428bb36b-b70a-4a2b-a5a1-1222b9cb1c4c None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:46.619895 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428bb36b-b70a-4a2b-a5a1-1222b9cb1c4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:46.620439 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7028ca15-7531-4e37-ba06-60791f815b4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:46.622165 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7028ca15-7531-4e37-ba06-60791f815b4a service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:46.658424 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 857/1702] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:36:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:36:46.674144 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:46.683098 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-428bb36b-b70a-4a2b-a5a1-1222b9cb1c4c None None] MFA Rules not processed for user `7b8e301214be4ffd8c7d39bbddf36f34`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:36:46.685132 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-428bb36b-b70a-4a2b-a5a1-1222b9cb1c4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:46.864536 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 846/1703] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:36:46 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:36:47.039272 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0a0e3f-3810-457f-a13e-85ca474483c6 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:47.039721 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0a0e3f-3810-457f-a13e-85ca474483c6 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:47.040182 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0a0e3f-3810-457f-a13e-85ca474483c6 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:47.042681 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 858/1704] 10.222.0.99 () {54 vars in 912 bytes} [Thu Jul 14 15:36:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:36:47.056836 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0099cc-338e-456d-b266-e155f8da1857 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:47.057769 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0099cc-338e-456d-b266-e155f8da1857 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:47.058202 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0099cc-338e-456d-b266-e155f8da1857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:47.110096 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-0b0099cc-338e-456d-b266-e155f8da1857 None None] MFA Rules not processed for user `0d0261759f9743b1a13a760ca9159305`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:36:47.120989 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0b0099cc-338e-456d-b266-e155f8da1857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:47.141446 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 847/1705] 10.222.0.99 () {58 vars in 1009 bytes} [Thu Jul 14 15:36:47 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:36:47.380946 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0a0e3f-3810-457f-a13e-85ca474483c6 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:47.383224 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-4b0a0e3f-3810-457f-a13e-85ca474483c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:47.417651 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0a0e3f-3810-457f-a13e-85ca474483c6 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:47.436137 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6856f671-89a8-41de-bfde-9ebcb95f35f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:47.438150 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6856f671-89a8-41de-bfde-9ebcb95f35f2 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:47.439945 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6856f671-89a8-41de-bfde-9ebcb95f35f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:47.441848 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6856f671-89a8-41de-bfde-9ebcb95f35f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:47.480817 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6856f671-89a8-41de-bfde-9ebcb95f35f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:47.486297 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6856f671-89a8-41de-bfde-9ebcb95f35f2 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:47.509608 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 859/1706] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:36:47 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:36:49.053698 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0099cc-338e-456d-b266-e155f8da1857 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:49.066885 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0b0099cc-338e-456d-b266-e155f8da1857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:49.113308 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0099cc-338e-456d-b266-e155f8da1857 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:49.158020 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e33606d-5a7a-4c88-82b6-ab60f9cf819d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:49.158967 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e33606d-5a7a-4c88-82b6-ab60f9cf819d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:49.159283 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e33606d-5a7a-4c88-82b6-ab60f9cf819d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:49.159735 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e33606d-5a7a-4c88-82b6-ab60f9cf819d admin admin] PATH_INFO: `/v3/projects/856bb3ac33614385bcf33535a36f113e` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:49.171234 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e33606d-5a7a-4c88-82b6-ab60f9cf819d admin admin] RBAC: Authorizing `identity:delete_project(project_id=856bb3ac33614385bcf33535a36f113e)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:49.189670 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e33606d-5a7a-4c88-82b6-ab60f9cf819d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:49.243171 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4120b7-abdb-4905-b128-689900b6cf2f None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:49.244136 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4120b7-abdb-4905-b128-689900b6cf2f None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:49.244603 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4120b7-abdb-4905-b128-689900b6cf2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:49.344736 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-3f4120b7-abdb-4905-b128-689900b6cf2f None None] MFA Rules not processed for user `d01c301ea20d4acab8dff1148501a975`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:36:49.346415 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-3f4120b7-abdb-4905-b128-689900b6cf2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:49.578820 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2e33606d-5a7a-4c88-82b6-ab60f9cf819d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '856bb3ac33614385bcf33535a36f113e', 'request_id': 'req-2e33606d-5a7a-4c88-82b6-ab60f9cf819d'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:36:49.613147 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:49.620791 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2e33606d-5a7a-4c88-82b6-ab60f9cf819d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '856bb3ac33614385bcf33535a36f113e', 'request_id': 'req-2e33606d-5a7a-4c88-82b6-ab60f9cf819d'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:36:49.663210 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2e33606d-5a7a-4c88-82b6-ab60f9cf819d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '856bb3ac33614385bcf33535a36f113e', 'request_id': 'req-2e33606d-5a7a-4c88-82b6-ab60f9cf819d'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:36:49.717819 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2e33606d-5a7a-4c88-82b6-ab60f9cf819d admin admin] The token cache is being invalidate because project 856bb3ac33614385bcf33535a36f113e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 15:36:49.722383 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2e33606d-5a7a-4c88-82b6-ab60f9cf819d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:36:49.726764 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2e33606d-5a7a-4c88-82b6-ab60f9cf819d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:36:49.745997 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 848/1707] 10.222.0.99 () {58 vars in 1232 bytes} [Thu Jul 14 15:36:49 2022] DELETE /identity/v3/projects/856bb3ac33614385bcf33535a36f113e => generated 0 bytes in 693 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:36:49.752678 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dd1e38-f8c8-4105-af46-d50adf5a8b32 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:49.756284 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dd1e38-f8c8-4105-af46-d50adf5a8b32 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:49.759367 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dd1e38-f8c8-4105-af46-d50adf5a8b32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:49.791734 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 860/1708] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:36:49 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 546 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:36:49.795809 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4120b7-abdb-4905-b128-689900b6cf2f None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:49.810084 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-3f4120b7-abdb-4905-b128-689900b6cf2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:49.861610 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-f3dd1e38-f8c8-4105-af46-d50adf5a8b32 None None] MFA Rules not processed for user `0d0261759f9743b1a13a760ca9159305`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:36:49.862396 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f3dd1e38-f8c8-4105-af46-d50adf5a8b32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:49.874777 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4120b7-abdb-4905-b128-689900b6cf2f None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:50.159882 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 849/1709] 10.222.0.124 () {58 vars in 1016 bytes} [Thu Jul 14 15:36:49 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 413 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:36:50.160893 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081acedc-ce95-4048-9271-8b08a8028de9 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:50.161196 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081acedc-ce95-4048-9271-8b08a8028de9 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:50.164954 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081acedc-ce95-4048-9271-8b08a8028de9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:50.175029 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d06185-b5cd-4532-b95e-30444ce2bd72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:50.180363 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d06185-b5cd-4532-b95e-30444ce2bd72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:50.181088 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d06185-b5cd-4532-b95e-30444ce2bd72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:50.181602 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d06185-b5cd-4532-b95e-30444ce2bd72 admin admin] PATH_INFO: `/v3/users/bb6e604977ea4e9aab17b0b60e68be04` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:50.186175 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d06185-b5cd-4532-b95e-30444ce2bd72 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb6e604977ea4e9aab17b0b60e68be04)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:50.188959 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d06185-b5cd-4532-b95e-30444ce2bd72 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:50.307178 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-081acedc-ce95-4048-9271-8b08a8028de9 None None] MFA Rules not processed for user `46f52ad195a34123982d238ea351172d`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:36:50.308397 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-081acedc-ce95-4048-9271-8b08a8028de9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:50.331395 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-59d06185-b5cd-4532-b95e-30444ce2bd72 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb6e604977ea4e9aab17b0b60e68be04'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:36:50.367028 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-59d06185-b5cd-4532-b95e-30444ce2bd72 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb6e604977ea4e9aab17b0b60e68be04'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:36:50.399233 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:50.409552 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-59d06185-b5cd-4532-b95e-30444ce2bd72 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb6e604977ea4e9aab17b0b60e68be04'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:36:50.410918 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-59d06185-b5cd-4532-b95e-30444ce2bd72 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb6e604977ea4e9aab17b0b60e68be04'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:36:50.411306 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-59d06185-b5cd-4532-b95e-30444ce2bd72 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb6e604977ea4e9aab17b0b60e68be04'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:36:50.461502 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 861/1710] 10.222.0.99 () {58 vars in 1223 bytes} [Thu Jul 14 15:36:49 2022] DELETE /identity/v3/users/bb6e604977ea4e9aab17b0b60e68be04 => generated 0 bytes in 673 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:36:50.462704 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d06185-b5cd-4532-b95e-30444ce2bd72 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:50.467480 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-59d06185-b5cd-4532-b95e-30444ce2bd72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:50.643140 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 850/1711] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:36:50 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 483 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:36:50.663199 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081acedc-ce95-4048-9271-8b08a8028de9 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:50.666744 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-081acedc-ce95-4048-9271-8b08a8028de9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:50.715001 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d06185-b5cd-4532-b95e-30444ce2bd72 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:50.725122 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b89d27-5b97-4143-9236-3d33812311d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:50.727384 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b89d27-5b97-4143-9236-3d33812311d3 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:50.750609 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b89d27-5b97-4143-9236-3d33812311d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:50.750609 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b89d27-5b97-4143-9236-3d33812311d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:50.759571 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-b2b89d27-5b97-4143-9236-3d33812311d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:50.812618 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b89d27-5b97-4143-9236-3d33812311d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:50.821126 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b89d27-5b97-4143-9236-3d33812311d3 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:50.866038 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:50.900934 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 862/1712] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:36:50 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:36:51.056067 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081acedc-ce95-4048-9271-8b08a8028de9 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:51.073603 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b89d27-5b97-4143-9236-3d33812311d3 service neutron] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:51.075458 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-b2b89d27-5b97-4143-9236-3d33812311d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:51.513444 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b89d27-5b97-4143-9236-3d33812311d3 service neutron] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:51.525027 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ce70ae-36da-4baf-a3c1-ea99c66e678c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9b1c2622934c5185f3b5f1ea623bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:51.525882 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ce70ae-36da-4baf-a3c1-ea99c66e678c service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:51.526178 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ce70ae-36da-4baf-a3c1-ea99c66e678c service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:51.526481 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ce70ae-36da-4baf-a3c1-ea99c66e678c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:51.539475 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b17cb32-ea2d-4293-88d9-a136bd1b8aa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:51.540427 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b17cb32-ea2d-4293-88d9-a136bd1b8aa9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:51.540767 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b17cb32-ea2d-4293-88d9-a136bd1b8aa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:51.541106 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b17cb32-ea2d-4293-88d9-a136bd1b8aa9 admin admin] PATH_INFO: `/v3/projects/e7c68922b5dc4b028e3f0d386a3ff8b8` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:51.560662 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b17cb32-ea2d-4293-88d9-a136bd1b8aa9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7c68922b5dc4b028e3f0d386a3ff8b8)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:51.572841 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b17cb32-ea2d-4293-88d9-a136bd1b8aa9 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:51.802226 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:51.838783 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-2b17cb32-ea2d-4293-88d9-a136bd1b8aa9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7c68922b5dc4b028e3f0d386a3ff8b8', 'request_id': 'req-2b17cb32-ea2d-4293-88d9-a136bd1b8aa9'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:36:51.856958 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ce70ae-36da-4baf-a3c1-ea99c66e678c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:51.859123 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ce70ae-36da-4baf-a3c1-ea99c66e678c service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:51.863304 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-2b17cb32-ea2d-4293-88d9-a136bd1b8aa9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7c68922b5dc4b028e3f0d386a3ff8b8', 'request_id': 'req-2b17cb32-ea2d-4293-88d9-a136bd1b8aa9'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:36:51.910072 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 851/1713] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:36:50 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 1247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:36:51.917680 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-2b17cb32-ea2d-4293-88d9-a136bd1b8aa9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7c68922b5dc4b028e3f0d386a3ff8b8', 'request_id': 'req-2b17cb32-ea2d-4293-88d9-a136bd1b8aa9'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:36:51.969549 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-2b17cb32-ea2d-4293-88d9-a136bd1b8aa9 admin admin] The token cache is being invalidate because project e7c68922b5dc4b028e3f0d386a3ff8b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 15:36:51.978929 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-2b17cb32-ea2d-4293-88d9-a136bd1b8aa9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:36:51.981924 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-2b17cb32-ea2d-4293-88d9-a136bd1b8aa9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:36:51.994653 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 863/1714] 10.222.0.99 () {58 vars in 1232 bytes} [Thu Jul 14 15:36:51 2022] DELETE /identity/v3/projects/e7c68922b5dc4b028e3f0d386a3ff8b8 => generated 0 bytes in 921 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 15:36:52.002177 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0e3b70-1e35-413f-8686-92ea08368e55 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:52.002490 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0e3b70-1e35-413f-8686-92ea08368e55 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:52.002785 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0e3b70-1e35-413f-8686-92ea08368e55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:52.079265 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-0c0e3b70-1e35-413f-8686-92ea08368e55 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:36:52.088946 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0c0e3b70-1e35-413f-8686-92ea08368e55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:52.432523 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 852/1715] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:36:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 433 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:36:52.434022 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca452e4a-044d-4e63-bc9a-143f3e1f2196 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:52.434310 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca452e4a-044d-4e63-bc9a-143f3e1f2196 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:52.436244 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca452e4a-044d-4e63-bc9a-143f3e1f2196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:52.480573 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-ca452e4a-044d-4e63-bc9a-143f3e1f2196 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:36:52.485760 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-ca452e4a-044d-4e63-bc9a-143f3e1f2196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:52.518934 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 864/1716] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:36:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:36:52.523362 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0e3b70-1e35-413f-8686-92ea08368e55 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:52.554346 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0e3b70-1e35-413f-8686-92ea08368e55 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:52.560640 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e70310c-0d2a-483d-ad7d-48edc2ad8f90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:52.561715 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e70310c-0d2a-483d-ad7d-48edc2ad8f90 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:52.564569 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e70310c-0d2a-483d-ad7d-48edc2ad8f90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:52.565099 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e70310c-0d2a-483d-ad7d-48edc2ad8f90 admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:52.566053 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e70310c-0d2a-483d-ad7d-48edc2ad8f90 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:36:52.566538 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e70310c-0d2a-483d-ad7d-48edc2ad8f90 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:52.568420 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e70310c-0d2a-483d-ad7d-48edc2ad8f90 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:52.612461 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 853/1717] 10.222.0.99 () {58 vars in 1152 bytes} [Thu Jul 14 15:36:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:36:52.616927 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca452e4a-044d-4e63-bc9a-143f3e1f2196 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:52.630916 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca452e4a-044d-4e63-bc9a-143f3e1f2196 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:52.657177 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe5e795-006d-4138-a22b-07a060e45eec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:52.657877 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe5e795-006d-4138-a22b-07a060e45eec admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:52.658237 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe5e795-006d-4138-a22b-07a060e45eec admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:52.658525 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe5e795-006d-4138-a22b-07a060e45eec admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:52.659346 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe5e795-006d-4138-a22b-07a060e45eec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:52.661759 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe5e795-006d-4138-a22b-07a060e45eec admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:52.798328 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 865/1718] 10.222.0.99 () {60 vars in 1152 bytes} [Thu Jul 14 15:36:52 2022] POST /identity/v3/projects => generated 373 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 15:36:52.802271 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e70310c-0d2a-483d-ad7d-48edc2ad8f90 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:52.815401 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e70310c-0d2a-483d-ad7d-48edc2ad8f90 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:53.128042 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b0ae40-f948-46a4-9e71-2bf227cc55b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:53.128741 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b0ae40-f948-46a4-9e71-2bf227cc55b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:53.129030 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b0ae40-f948-46a4-9e71-2bf227cc55b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:53.129310 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b0ae40-f948-46a4-9e71-2bf227cc55b7 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:53.130024 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b0ae40-f948-46a4-9e71-2bf227cc55b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:53.137104 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b0ae40-f948-46a4-9e71-2bf227cc55b7 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:53.199869 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 854/1719] 10.222.0.99 () {60 vars in 1143 bytes} [Thu Jul 14 15:36:52 2022] POST /identity/v3/users => generated 287 bytes in 398 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 15:36:53.201712 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe5e795-006d-4138-a22b-07a060e45eec admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:53.241126 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe5e795-006d-4138-a22b-07a060e45eec admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:53.277447 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2439f32e-674b-405f-8c58-d73019d179c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:53.281638 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2439f32e-674b-405f-8c58-d73019d179c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:53.283566 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2439f32e-674b-405f-8c58-d73019d179c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:53.288407 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2439f32e-674b-405f-8c58-d73019d179c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:53.291834 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2439f32e-674b-405f-8c58-d73019d179c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:53.296693 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2439f32e-674b-405f-8c58-d73019d179c8 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:53.319190 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 866/1720] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:36:53 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:36:53.326466 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b0ae40-f948-46a4-9e71-2bf227cc55b7 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:53.357198 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b0ae40-f948-46a4-9e71-2bf227cc55b7 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:53.366257 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6d786a-d890-4b22-aa4e-49c6660bcd4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:53.368003 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6d786a-d890-4b22-aa4e-49c6660bcd4b admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:53.368297 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6d786a-d890-4b22-aa4e-49c6660bcd4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:53.369132 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6d786a-d890-4b22-aa4e-49c6660bcd4b admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:53.371535 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6d786a-d890-4b22-aa4e-49c6660bcd4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:53.373684 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6d786a-d890-4b22-aa4e-49c6660bcd4b admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:53.427016 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 855/1721] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:36:53 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:36:53.431347 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2439f32e-674b-405f-8c58-d73019d179c8 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:53.470278 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2439f32e-674b-405f-8c58-d73019d179c8 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:53.511707 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aaed48-f65d-4371-9391-b7f1dbf30d6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:53.513691 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aaed48-f65d-4371-9391-b7f1dbf30d6d admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:53.514358 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aaed48-f65d-4371-9391-b7f1dbf30d6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:53.515689 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aaed48-f65d-4371-9391-b7f1dbf30d6d admin admin] PATH_INFO: `/v3/projects/a6c04cbbcc13499585afd5f799cc63aa/users/aefc8029ae754438ad70672adf22b3c1/roles/b8c98fddc0fd4436b577fd1d3361bc38` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:53.571392 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62aaed48-f65d-4371-9391-b7f1dbf30d6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6c04cbbcc13499585afd5f799cc63aa, user_id=aefc8029ae754438ad70672adf22b3c1, role_id=b8c98fddc0fd4436b577fd1d3361bc38)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:53.576717 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62aaed48-f65d-4371-9391-b7f1dbf30d6d admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:53.606934 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 867/1722] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 15:36:53 2022] PUT /identity/v3/projects/a6c04cbbcc13499585afd5f799cc63aa/users/aefc8029ae754438ad70672adf22b3c1/roles/b8c98fddc0fd4436b577fd1d3361bc38 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 15:36:53.621304 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e3d872-9063-4715-bd2a-016a016bc983 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:53.622959 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e3d872-9063-4715-bd2a-016a016bc983 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:53.623260 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e3d872-9063-4715-bd2a-016a016bc983 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:53.691245 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-c7e3d872-9063-4715-bd2a-016a016bc983 None None] MFA Rules not processed for user `aefc8029ae754438ad70672adf22b3c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:36:53.694715 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c7e3d872-9063-4715-bd2a-016a016bc983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:53.890912 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:54.021120 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 856/1723] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:36:53 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 403 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:36:54.025689 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aaed48-f65d-4371-9391-b7f1dbf30d6d admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:54.116576 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:54.131695 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9da44bf-9482-41c8-aef0-2ac15657b692 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:54.132179 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9da44bf-9482-41c8-aef0-2ac15657b692 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:54.132770 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9da44bf-9482-41c8-aef0-2ac15657b692 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:54.135117 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 857/1724] 10.222.0.99 () {54 vars in 921 bytes} [Thu Jul 14 15:36:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:36:54.143813 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3668a3de-58cf-4244-b66f-e626c885e7af None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:54.146056 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3668a3de-58cf-4244-b66f-e626c885e7af None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:54.146453 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3668a3de-58cf-4244-b66f-e626c885e7af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:54.271879 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-3668a3de-58cf-4244-b66f-e626c885e7af None None] MFA Rules not processed for user `dc157c485f794e968a60d8405c9310ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:36:54.291101 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-3668a3de-58cf-4244-b66f-e626c885e7af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:54.376281 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aaed48-f65d-4371-9391-b7f1dbf30d6d admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:54.403243 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a67ee9-4916-40c2-8dff-9faace996dcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:54.406345 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a67ee9-4916-40c2-8dff-9faace996dcb admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:54.406656 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a67ee9-4916-40c2-8dff-9faace996dcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:54.409304 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a67ee9-4916-40c2-8dff-9faace996dcb admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:54.410374 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a67ee9-4916-40c2-8dff-9faace996dcb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:54.418066 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a67ee9-4916-40c2-8dff-9faace996dcb admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:54.570014 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 858/1725] 10.222.0.99 () {58 vars in 1018 bytes} [Thu Jul 14 15:36:54 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 427 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:36:54.574898 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3668a3de-58cf-4244-b66f-e626c885e7af None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:54.576610 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-3668a3de-58cf-4244-b66f-e626c885e7af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:54.606892 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 868/1726] 10.222.0.99 () {60 vars in 1152 bytes} [Thu Jul 14 15:36:54 2022] POST /identity/v3/projects => generated 371 bytes in 582 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 15:36:54.612059 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a67ee9-4916-40c2-8dff-9faace996dcb admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:54.618670 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3668a3de-58cf-4244-b66f-e626c885e7af None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:54.765260 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96864ad6-29d5-40d5-bb57-860c66f1c0fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:54.766331 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96864ad6-29d5-40d5-bb57-860c66f1c0fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:54.766991 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96864ad6-29d5-40d5-bb57-860c66f1c0fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:54.766991 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96864ad6-29d5-40d5-bb57-860c66f1c0fd admin admin] PATH_INFO: `/v3/projects/1616277137814fda8ac82eea4fe078e4` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:54.773029 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96864ad6-29d5-40d5-bb57-860c66f1c0fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=1616277137814fda8ac82eea4fe078e4)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:54.776144 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96864ad6-29d5-40d5-bb57-860c66f1c0fd admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:54.911100 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-96864ad6-29d5-40d5-bb57-860c66f1c0fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1616277137814fda8ac82eea4fe078e4', 'request_id': 'req-96864ad6-29d5-40d5-bb57-860c66f1c0fd'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:36:54.923286 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-96864ad6-29d5-40d5-bb57-860c66f1c0fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1616277137814fda8ac82eea4fe078e4', 'request_id': 'req-96864ad6-29d5-40d5-bb57-860c66f1c0fd'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:36:54.937886 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-96864ad6-29d5-40d5-bb57-860c66f1c0fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1616277137814fda8ac82eea4fe078e4', 'request_id': 'req-96864ad6-29d5-40d5-bb57-860c66f1c0fd'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:36:54.960728 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-96864ad6-29d5-40d5-bb57-860c66f1c0fd admin admin] The token cache is being invalidate because project 1616277137814fda8ac82eea4fe078e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 15:36:54.961180 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-96864ad6-29d5-40d5-bb57-860c66f1c0fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:36:54.962472 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-96864ad6-29d5-40d5-bb57-860c66f1c0fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:36:54.964719 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 859/1727] 10.222.0.99 () {58 vars in 1232 bytes} [Thu Jul 14 15:36:54 2022] DELETE /identity/v3/projects/1616277137814fda8ac82eea4fe078e4 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:36:54.967535 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96864ad6-29d5-40d5-bb57-860c66f1c0fd admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:54.970637 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-96864ad6-29d5-40d5-bb57-860c66f1c0fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:54.980741 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a67ee9-4916-40c2-8dff-9faace996dcb admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:54.999778 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:55.178859 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5fd0a2-0b95-498f-ac01-d845d5026d01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:55.179675 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5fd0a2-0b95-498f-ac01-d845d5026d01 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:55.180648 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5fd0a2-0b95-498f-ac01-d845d5026d01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:55.181006 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5fd0a2-0b95-498f-ac01-d845d5026d01 admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:55.181834 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5fd0a2-0b95-498f-ac01-d845d5026d01 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:55.184179 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5fd0a2-0b95-498f-ac01-d845d5026d01 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:55.203866 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96864ad6-29d5-40d5-bb57-860c66f1c0fd admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:55.218330 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a3f54e-dd5c-4384-9ff7-2eeae66f97b1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc157c485f794e968a60d8405c9310ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:55.219027 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a3f54e-dd5c-4384-9ff7-2eeae66f97b1 service placement] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:55.219319 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a3f54e-dd5c-4384-9ff7-2eeae66f97b1 service placement] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:55.219621 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a3f54e-dd5c-4384-9ff7-2eeae66f97b1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:55.234489 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b6a3f54e-dd5c-4384-9ff7-2eeae66f97b1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:55.246462 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 869/1728] 10.222.0.99 () {60 vars in 1143 bytes} [Thu Jul 14 15:36:54 2022] POST /identity/v3/users => generated 292 bytes in 635 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:36:55.251804 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840140a5-afa0-44da-ab3a-d18243a63c7e None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:55.252797 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840140a5-afa0-44da-ab3a-d18243a63c7e None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:55.253261 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840140a5-afa0-44da-ab3a-d18243a63c7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:55.267312 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a3f54e-dd5c-4384-9ff7-2eeae66f97b1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:55.272072 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a3f54e-dd5c-4384-9ff7-2eeae66f97b1 service placement] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:55.300292 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-840140a5-afa0-44da-ab3a-d18243a63c7e None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:36:55.303694 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-840140a5-afa0-44da-ab3a-d18243a63c7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:55.313738 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 860/1729] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:36:54 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:36:55.317877 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a3f54e-dd5c-4384-9ff7-2eeae66f97b1 service placement] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:55.326811 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 870/1730] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:36:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:36:55.327398 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b6a3f54e-dd5c-4384-9ff7-2eeae66f97b1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:55.335515 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e7349d-2120-4d58-a929-5394a653dddf None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:55.336574 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e7349d-2120-4d58-a929-5394a653dddf None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:55.337859 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e7349d-2120-4d58-a929-5394a653dddf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:55.371442 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a3f54e-dd5c-4384-9ff7-2eeae66f97b1 service placement] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:55.386750 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22143187-85f7-4307-ab00-77e85fcff8b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:55.388001 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22143187-85f7-4307-ab00-77e85fcff8b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:55.388671 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22143187-85f7-4307-ab00-77e85fcff8b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:55.389208 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22143187-85f7-4307-ab00-77e85fcff8b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:55.392350 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22143187-85f7-4307-ab00-77e85fcff8b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:55.395332 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22143187-85f7-4307-ab00-77e85fcff8b8 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:55.460409 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 861/1731] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:36:55 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:36:55.465712 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22143187-85f7-4307-ab00-77e85fcff8b8 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:55.486736 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-40e7349d-2120-4d58-a929-5394a653dddf None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:36:55.489019 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22143187-85f7-4307-ab00-77e85fcff8b8 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:55.494708 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-40e7349d-2120-4d58-a929-5394a653dddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:55.496460 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a9f250-d45f-49e4-a846-80c18d87fa7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:55.497656 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a9f250-d45f-49e4-a846-80c18d87fa7c admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:55.498239 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a9f250-d45f-49e4-a846-80c18d87fa7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:55.498659 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a9f250-d45f-49e4-a846-80c18d87fa7c admin admin] PATH_INFO: `/v3/projects/7e8d26e151614d86b9cda47276c6f685/users/a02d99f63ecf4053afb8aa04eccef484/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:55.535523 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a9f250-d45f-49e4-a846-80c18d87fa7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e8d26e151614d86b9cda47276c6f685, user_id=a02d99f63ecf4053afb8aa04eccef484, role_id=d0f6ba1314f14d9f9c816bdb5d0c3aac)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:55.538855 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a9f250-d45f-49e4-a846-80c18d87fa7c admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:55.549574 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 871/1732] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:36:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:36:55.552236 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e7349d-2120-4d58-a929-5394a653dddf None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:55.566682 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 862/1733] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 15:36:55 2022] PUT /identity/v3/projects/7e8d26e151614d86b9cda47276c6f685/users/a02d99f63ecf4053afb8aa04eccef484/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 15:36:55.572356 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a9f250-d45f-49e4-a846-80c18d87fa7c admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:55.609868 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e7349d-2120-4d58-a929-5394a653dddf None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:55.745683 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:55.758518 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a9f250-d45f-49e4-a846-80c18d87fa7c admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:55.786247 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2eabe4f-30ac-4fec-b9a6-7641e1a0bccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:55.791573 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eabe4f-30ac-4fec-b9a6-7641e1a0bccf admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:55.792763 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eabe4f-30ac-4fec-b9a6-7641e1a0bccf admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:55.793995 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eabe4f-30ac-4fec-b9a6-7641e1a0bccf admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:55.797998 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2eabe4f-30ac-4fec-b9a6-7641e1a0bccf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:55.806226 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2eabe4f-30ac-4fec-b9a6-7641e1a0bccf admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:55.911769 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 863/1734] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:36:55 2022] GET /identity/v3/roles => generated 1420 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:36:55.913652 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbb3661-e205-4d62-a0eb-df90ca905d6a None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:55.914895 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3977e106-5e3e-4a8a-9cae-ac6993534d03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:55.917396 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3977e106-5e3e-4a8a-9cae-ac6993534d03 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:55.918814 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbb3661-e205-4d62-a0eb-df90ca905d6a None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:55.922415 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3977e106-5e3e-4a8a-9cae-ac6993534d03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:55.922921 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbb3661-e205-4d62-a0eb-df90ca905d6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:55.924803 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3977e106-5e3e-4a8a-9cae-ac6993534d03 admin admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:55.929482 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3977e106-5e3e-4a8a-9cae-ac6993534d03 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:36:55.930801 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3977e106-5e3e-4a8a-9cae-ac6993534d03 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:55.946759 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3977e106-5e3e-4a8a-9cae-ac6993534d03 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:56.054827 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 872/1735] 10.222.0.99 () {58 vars in 1152 bytes} [Thu Jul 14 15:36:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 504 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:36:56.057323 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3977e106-5e3e-4a8a-9cae-ac6993534d03 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:56.066941 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-efbb3661-e205-4d62-a0eb-df90ca905d6a None None] MFA Rules not processed for user `781567471d0f4ca4add4331d405febd4`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:36:56.074052 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-efbb3661-e205-4d62-a0eb-df90ca905d6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:56.121491 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3977e106-5e3e-4a8a-9cae-ac6993534d03 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:56.143178 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f306b58-b9ae-47d1-92a3-e1943be560e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:56.145111 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f306b58-b9ae-47d1-92a3-e1943be560e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:56.146504 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f306b58-b9ae-47d1-92a3-e1943be560e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:56.147143 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f306b58-b9ae-47d1-92a3-e1943be560e8 admin admin] PATH_INFO: `/v3/projects/7e8d26e151614d86b9cda47276c6f685/users/a02d99f63ecf4053afb8aa04eccef484/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:56.150317 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f306b58-b9ae-47d1-92a3-e1943be560e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e8d26e151614d86b9cda47276c6f685, user_id=a02d99f63ecf4053afb8aa04eccef484, role_id=d0f6ba1314f14d9f9c816bdb5d0c3aac)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:56.153418 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f306b58-b9ae-47d1-92a3-e1943be560e8 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:56.205266 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 873/1736] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 15:36:56 2022] PUT /identity/v3/projects/7e8d26e151614d86b9cda47276c6f685/users/a02d99f63ecf4053afb8aa04eccef484/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:36:56.208410 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f306b58-b9ae-47d1-92a3-e1943be560e8 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:56.339100 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 864/1737] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:36:55 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 426 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:36:56.346165 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096b04dd-5961-471f-8356-6b04b0f3afd7 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:56.346452 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096b04dd-5961-471f-8356-6b04b0f3afd7 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:56.346735 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096b04dd-5961-471f-8356-6b04b0f3afd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:56.353019 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f306b58-b9ae-47d1-92a3-e1943be560e8 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:56.361347 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda5f3ba-e026-4dc7-aafc-7ec675e78b54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:56.362432 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda5f3ba-e026-4dc7-aafc-7ec675e78b54 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:56.363119 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda5f3ba-e026-4dc7-aafc-7ec675e78b54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:56.363652 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda5f3ba-e026-4dc7-aafc-7ec675e78b54 admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:56.364616 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda5f3ba-e026-4dc7-aafc-7ec675e78b54 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:56.367030 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda5f3ba-e026-4dc7-aafc-7ec675e78b54 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:56.447457 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:56.470240 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 874/1738] 10.222.0.99 () {60 vars in 1152 bytes} [Thu Jul 14 15:36:56 2022] POST /identity/v3/projects => generated 365 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:36:56.477325 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5887dce9-8bb3-4842-990a-cf01d13678ac None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:56.482542 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5887dce9-8bb3-4842-990a-cf01d13678ac None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:56.483048 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5887dce9-8bb3-4842-990a-cf01d13678ac None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:56.487459 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 875/1739] 10.222.0.99 () {54 vars in 921 bytes} [Thu Jul 14 15:36:56 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:36:56.495558 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5887dce9-8bb3-4842-990a-cf01d13678ac None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:56.540635 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-096b04dd-5961-471f-8356-6b04b0f3afd7 None None] MFA Rules not processed for user `a02d99f63ecf4053afb8aa04eccef484`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:36:56.556911 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-096b04dd-5961-471f-8356-6b04b0f3afd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:56.745788 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5887dce9-8bb3-4842-990a-cf01d13678ac None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:56.763572 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d334cb8-c7e5-432b-b3dd-2b0acf4ac5b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:56.765023 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d334cb8-c7e5-432b-b3dd-2b0acf4ac5b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:56.765316 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d334cb8-c7e5-432b-b3dd-2b0acf4ac5b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:56.765940 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d334cb8-c7e5-432b-b3dd-2b0acf4ac5b6 admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:56.766457 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d334cb8-c7e5-432b-b3dd-2b0acf4ac5b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:56.777990 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d334cb8-c7e5-432b-b3dd-2b0acf4ac5b6 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:56.851523 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 876/1740] 10.222.0.99 () {60 vars in 1143 bytes} [Thu Jul 14 15:36:56 2022] POST /identity/v3/users => generated 283 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:36:56.851754 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 865/1741] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:36:56 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 505 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:36:56.854803 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170e50b9-96bc-4961-81ab-250e33cad8b6 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:56.864193 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096b04dd-5961-471f-8356-6b04b0f3afd7 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:56.865521 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170e50b9-96bc-4961-81ab-250e33cad8b6 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:56.867003 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170e50b9-96bc-4961-81ab-250e33cad8b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:56.886687 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096b04dd-5961-471f-8356-6b04b0f3afd7 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:56.901948 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86cc730-614e-47c3-93ff-9ea41acf5b1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:56.903370 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86cc730-614e-47c3-93ff-9ea41acf5b1d admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:56.903837 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86cc730-614e-47c3-93ff-9ea41acf5b1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:56.904339 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86cc730-614e-47c3-93ff-9ea41acf5b1d admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:56.907922 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86cc730-614e-47c3-93ff-9ea41acf5b1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:56.910247 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86cc730-614e-47c3-93ff-9ea41acf5b1d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:56.911571 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-170e50b9-96bc-4961-81ab-250e33cad8b6 None None] MFA Rules not processed for user `3a9b1c2622934c5185f3b5f1ea623bc0`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:36:56.915958 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-170e50b9-96bc-4961-81ab-250e33cad8b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:56.950624 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 866/1742] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:36:56 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:36:56.954523 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86cc730-614e-47c3-93ff-9ea41acf5b1d admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:56.960701 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d86cc730-614e-47c3-93ff-9ea41acf5b1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:57.005664 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:57.162941 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 877/1743] 10.222.0.99 () {58 vars in 1018 bytes} [Thu Jul 14 15:36:56 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:36:57.163865 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170e50b9-96bc-4961-81ab-250e33cad8b6 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:57.172899 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86cc730-614e-47c3-93ff-9ea41acf5b1d admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:57.200000 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3833b973-9acc-4ba1-838b-e3e96018dcac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3e1a2b83f3d4d88ba5084dfaffd22f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:57.206608 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3833b973-9acc-4ba1-838b-e3e96018dcac service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:57.206693 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3833b973-9acc-4ba1-838b-e3e96018dcac service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:57.207002 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3833b973-9acc-4ba1-838b-e3e96018dcac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:57.224619 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170e50b9-96bc-4961-81ab-250e33cad8b6 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:57.257661 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5ee54c-2c9d-4d47-8741-c0c83d0d8010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:57.258380 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5ee54c-2c9d-4d47-8741-c0c83d0d8010 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:57.258669 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5ee54c-2c9d-4d47-8741-c0c83d0d8010 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:57.258966 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5ee54c-2c9d-4d47-8741-c0c83d0d8010 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:57.259731 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5ee54c-2c9d-4d47-8741-c0c83d0d8010 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:57.259826 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3833b973-9acc-4ba1-838b-e3e96018dcac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:57.261382 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5ee54c-2c9d-4d47-8741-c0c83d0d8010 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:57.271425 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3833b973-9acc-4ba1-838b-e3e96018dcac service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:57.309785 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 878/1744] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:36:57 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:36:57.316936 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5ee54c-2c9d-4d47-8741-c0c83d0d8010 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:57.322761 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 867/1745] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:36:56 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:36:57.358643 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5ee54c-2c9d-4d47-8741-c0c83d0d8010 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:57.389215 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15944825-fe50-4059-8725-0da352fea474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:57.398525 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15944825-fe50-4059-8725-0da352fea474 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:57.398796 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15944825-fe50-4059-8725-0da352fea474 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:57.399129 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15944825-fe50-4059-8725-0da352fea474 admin admin] PATH_INFO: `/v3/projects/9411879468964f759dd0ba0d159dfa9f/users/1d641b71e8c54f2188f19e538570a9c2/roles/b8c98fddc0fd4436b577fd1d3361bc38` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:57.465608 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15944825-fe50-4059-8725-0da352fea474 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9411879468964f759dd0ba0d159dfa9f, user_id=1d641b71e8c54f2188f19e538570a9c2, role_id=b8c98fddc0fd4436b577fd1d3361bc38)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:57.475934 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15944825-fe50-4059-8725-0da352fea474 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:57.534648 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 879/1746] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 15:36:57 2022] PUT /identity/v3/projects/9411879468964f759dd0ba0d159dfa9f/users/1d641b71e8c54f2188f19e538570a9c2/roles/b8c98fddc0fd4436b577fd1d3361bc38 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:36:57.546652 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41998efb-9f90-4360-b9a9-6232feec11f6 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:57.547207 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41998efb-9f90-4360-b9a9-6232feec11f6 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:57.547901 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41998efb-9f90-4360-b9a9-6232feec11f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:57.638966 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-41998efb-9f90-4360-b9a9-6232feec11f6 None None] MFA Rules not processed for user `1d641b71e8c54f2188f19e538570a9c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:36:57.646889 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-41998efb-9f90-4360-b9a9-6232feec11f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:57.839303 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 868/1747] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:36:57 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:36:57.844365 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15944825-fe50-4059-8725-0da352fea474 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:58.015154 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41998efb-9f90-4360-b9a9-6232feec11f6 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:58.016794 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-41998efb-9f90-4360-b9a9-6232feec11f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:58.057048 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15944825-fe50-4059-8725-0da352fea474 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:58.059283 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:58.073105 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f7fbcc-9a51-4bb7-ac60-c393f0ad7956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:58.073803 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f7fbcc-9a51-4bb7-ac60-c393f0ad7956 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:58.074093 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f7fbcc-9a51-4bb7-ac60-c393f0ad7956 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:58.074377 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f7fbcc-9a51-4bb7-ac60-c393f0ad7956 admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:58.075101 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f7fbcc-9a51-4bb7-ac60-c393f0ad7956 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:58.078062 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f7fbcc-9a51-4bb7-ac60-c393f0ad7956 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:58.159488 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 880/1748] 10.222.0.99 () {60 vars in 1152 bytes} [Thu Jul 14 15:36:57 2022] POST /identity/v3/projects => generated 367 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:36:58.164489 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f7fbcc-9a51-4bb7-ac60-c393f0ad7956 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:58.247613 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41998efb-9f90-4360-b9a9-6232feec11f6 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:58.434242 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f7fbcc-9a51-4bb7-ac60-c393f0ad7956 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:58.455333 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277e8ba8-eb43-4753-b897-4c27e61a9f8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:58.457486 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277e8ba8-eb43-4753-b897-4c27e61a9f8a admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:58.459934 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277e8ba8-eb43-4753-b897-4c27e61a9f8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:58.461114 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277e8ba8-eb43-4753-b897-4c27e61a9f8a admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:58.462517 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277e8ba8-eb43-4753-b897-4c27e61a9f8a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:58.473016 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277e8ba8-eb43-4753-b897-4c27e61a9f8a admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:58.498756 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0570c9-c483-463c-8fcb-3964e8d0d714 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9b1c2622934c5185f3b5f1ea623bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:58.500611 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0570c9-c483-463c-8fcb-3964e8d0d714 service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:58.501235 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0570c9-c483-463c-8fcb-3964e8d0d714 service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:58.504015 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0570c9-c483-463c-8fcb-3964e8d0d714 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:58.525395 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 881/1749] 10.222.0.99 () {60 vars in 1143 bytes} [Thu Jul 14 15:36:58 2022] POST /identity/v3/users => generated 290 bytes in 361 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:36:58.528015 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277e8ba8-eb43-4753-b897-4c27e61a9f8a admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:58.538491 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:58.544566 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0570c9-c483-463c-8fcb-3964e8d0d714 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:58.548201 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0570c9-c483-463c-8fcb-3964e8d0d714 service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:58.557423 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277e8ba8-eb43-4753-b897-4c27e61a9f8a admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:58.565296 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d592b6f9-3e77-4680-89b0-f49c967ade3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:58.566678 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d592b6f9-3e77-4680-89b0-f49c967ade3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:58.567183 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d592b6f9-3e77-4680-89b0-f49c967ade3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:58.567808 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d592b6f9-3e77-4680-89b0-f49c967ade3f admin admin] PATH_INFO: `/v3/users/aefc8029ae754438ad70672adf22b3c1` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:58.569556 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d592b6f9-3e77-4680-89b0-f49c967ade3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=aefc8029ae754438ad70672adf22b3c1)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:58.572130 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d592b6f9-3e77-4680-89b0-f49c967ade3f admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:58.589418 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 869/1750] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:36:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 574 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 15:36:58.595100 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0570c9-c483-463c-8fcb-3964e8d0d714 service nova] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:58.667691 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0570c9-c483-463c-8fcb-3964e8d0d714 service nova] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:58.689914 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f23abe7-fd28-4785-aebd-faaaae9b650d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:58.690672 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f23abe7-fd28-4785-aebd-faaaae9b650d admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:58.690967 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f23abe7-fd28-4785-aebd-faaaae9b650d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:58.691251 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f23abe7-fd28-4785-aebd-faaaae9b650d admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:58.701470 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f23abe7-fd28-4785-aebd-faaaae9b650d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:58.712395 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f23abe7-fd28-4785-aebd-faaaae9b650d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:58.751547 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-d592b6f9-3e77-4680-89b0-f49c967ade3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aefc8029ae754438ad70672adf22b3c1'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:36:58.762487 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 870/1751] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:36:58 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:36:58.763748 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-d592b6f9-3e77-4680-89b0-f49c967ade3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aefc8029ae754438ad70672adf22b3c1'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:36:58.769389 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f23abe7-fd28-4785-aebd-faaaae9b650d admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:58.778850 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-7f23abe7-fd28-4785-aebd-faaaae9b650d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:58.801119 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-d592b6f9-3e77-4680-89b0-f49c967ade3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aefc8029ae754438ad70672adf22b3c1'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:36:58.803124 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-d592b6f9-3e77-4680-89b0-f49c967ade3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aefc8029ae754438ad70672adf22b3c1'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:36:58.809216 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-d592b6f9-3e77-4680-89b0-f49c967ade3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aefc8029ae754438ad70672adf22b3c1'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:36:58.819122 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f23abe7-fd28-4785-aebd-faaaae9b650d admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:58.833930 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef9490c-67b8-4de7-85de-15675349610e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:58.834841 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef9490c-67b8-4de7-85de-15675349610e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:58.835133 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef9490c-67b8-4de7-85de-15675349610e admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:58.835415 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef9490c-67b8-4de7-85de-15675349610e admin admin] PATH_INFO: `/v3/users/1c43928aa8cf4d729873ae9361dcb532` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:58.838108 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef9490c-67b8-4de7-85de-15675349610e admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c43928aa8cf4d729873ae9361dcb532)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:58.840099 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef9490c-67b8-4de7-85de-15675349610e admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:58.847380 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 882/1752] 10.222.0.99 () {58 vars in 1223 bytes} [Thu Jul 14 15:36:58 2022] DELETE /identity/v3/users/aefc8029ae754438ad70672adf22b3c1 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:36:58.849156 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d592b6f9-3e77-4680-89b0-f49c967ade3f admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:58.850283 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-d592b6f9-3e77-4680-89b0-f49c967ade3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:58.976033 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2ef9490c-67b8-4de7-85de-15675349610e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c43928aa8cf4d729873ae9361dcb532'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:36:59.003390 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2ef9490c-67b8-4de7-85de-15675349610e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c43928aa8cf4d729873ae9361dcb532'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:36:59.030840 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2ef9490c-67b8-4de7-85de-15675349610e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c43928aa8cf4d729873ae9361dcb532'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:36:59.031633 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2ef9490c-67b8-4de7-85de-15675349610e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c43928aa8cf4d729873ae9361dcb532'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:36:59.033025 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2ef9490c-67b8-4de7-85de-15675349610e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c43928aa8cf4d729873ae9361dcb532'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:36:59.057102 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d592b6f9-3e77-4680-89b0-f49c967ade3f admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:59.059798 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:59.074053 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 871/1753] 10.222.0.99 () {58 vars in 1223 bytes} [Thu Jul 14 15:36:58 2022] DELETE /identity/v3/users/1c43928aa8cf4d729873ae9361dcb532 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:36:59.077289 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e61816-ae8c-45c1-ad33-7e0bd2a9821a None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:59.077420 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6561fd60-0919-413e-a967-616ebeae4c65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:59.077754 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6561fd60-0919-413e-a967-616ebeae4c65 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:59.078048 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6561fd60-0919-413e-a967-616ebeae4c65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:59.078332 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6561fd60-0919-413e-a967-616ebeae4c65 admin admin] PATH_INFO: `/v3/projects/76505a2a22534241b3d8f5c8252c7e71/users/da90d91e159f4d31b7f2506961bb6d5e/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:59.079291 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e61816-ae8c-45c1-ad33-7e0bd2a9821a None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:59.079661 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e61816-ae8c-45c1-ad33-7e0bd2a9821a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:59.117398 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6561fd60-0919-413e-a967-616ebeae4c65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76505a2a22534241b3d8f5c8252c7e71, user_id=da90d91e159f4d31b7f2506961bb6d5e, role_id=d0f6ba1314f14d9f9c816bdb5d0c3aac)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:59.119662 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-71e61816-ae8c-45c1-ad33-7e0bd2a9821a None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:36:59.122660 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6561fd60-0919-413e-a967-616ebeae4c65 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:59.125119 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-71e61816-ae8c-45c1-ad33-7e0bd2a9821a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:59.150687 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 883/1754] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 15:36:58 2022] PUT /identity/v3/projects/76505a2a22534241b3d8f5c8252c7e71/users/da90d91e159f4d31b7f2506961bb6d5e/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:36:59.161518 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6561fd60-0919-413e-a967-616ebeae4c65 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:59.166944 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-6561fd60-0919-413e-a967-616ebeae4c65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:59.372463 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 872/1755] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:36:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:36:59.402446 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:59.409754 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6561fd60-0919-413e-a967-616ebeae4c65 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:59.422560 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e61816-ae8c-45c1-ad33-7e0bd2a9821a None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:59.424061 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-71e61816-ae8c-45c1-ad33-7e0bd2a9821a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:59.455620 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82bd44a-8dfe-4270-a48f-9c6a3162acf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:59.456346 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82bd44a-8dfe-4270-a48f-9c6a3162acf4 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:59.456658 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82bd44a-8dfe-4270-a48f-9c6a3162acf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:59.456994 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82bd44a-8dfe-4270-a48f-9c6a3162acf4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:59.458216 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82bd44a-8dfe-4270-a48f-9c6a3162acf4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:59.465749 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82bd44a-8dfe-4270-a48f-9c6a3162acf4 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:59.496835 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 884/1756] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:36:59 2022] GET /identity/v3/roles => generated 1420 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:36:59.504672 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82bd44a-8dfe-4270-a48f-9c6a3162acf4 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:59.539461 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82bd44a-8dfe-4270-a48f-9c6a3162acf4 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:59.554067 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e061b6fe-1d18-4f41-8b52-0691419d9e68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:59.554998 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e061b6fe-1d18-4f41-8b52-0691419d9e68 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:59.555292 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e061b6fe-1d18-4f41-8b52-0691419d9e68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:59.555584 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e061b6fe-1d18-4f41-8b52-0691419d9e68 admin admin] PATH_INFO: `/v3/projects/76505a2a22534241b3d8f5c8252c7e71/users/da90d91e159f4d31b7f2506961bb6d5e/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:59.560055 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e061b6fe-1d18-4f41-8b52-0691419d9e68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76505a2a22534241b3d8f5c8252c7e71, user_id=da90d91e159f4d31b7f2506961bb6d5e, role_id=d0f6ba1314f14d9f9c816bdb5d0c3aac)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:59.563276 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e061b6fe-1d18-4f41-8b52-0691419d9e68 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:59.611079 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 885/1757] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 15:36:59 2022] PUT /identity/v3/projects/76505a2a22534241b3d8f5c8252c7e71/users/da90d91e159f4d31b7f2506961bb6d5e/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:36:59.621838 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dd39f5-fa4c-4413-8cef-9c1fbab09ee1 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:59.622438 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dd39f5-fa4c-4413-8cef-9c1fbab09ee1 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:59.622901 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dd39f5-fa4c-4413-8cef-9c1fbab09ee1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:59.655938 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e61816-ae8c-45c1-ad33-7e0bd2a9821a None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:59.735645 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-c0dd39f5-fa4c-4413-8cef-9c1fbab09ee1 None None] MFA Rules not processed for user `da90d91e159f4d31b7f2506961bb6d5e`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:36:59.739735 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-c0dd39f5-fa4c-4413-8cef-9c1fbab09ee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:59.766808 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b286b7-b03f-438f-8510-a7f2d76f805e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:59.767735 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b286b7-b03f-438f-8510-a7f2d76f805e service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:59.768303 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b286b7-b03f-438f-8510-a7f2d76f805e service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:59.768696 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b286b7-b03f-438f-8510-a7f2d76f805e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:59.990721 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b286b7-b03f-438f-8510-a7f2d76f805e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:59.992380 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 886/1758] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:36:59 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 371 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:37:00.005342 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dd39f5-fa4c-4413-8cef-9c1fbab09ee1 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:00.006712 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-c0dd39f5-fa4c-4413-8cef-9c1fbab09ee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:00.008556 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b286b7-b03f-438f-8510-a7f2d76f805e service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:00.035046 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 873/1759] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:36:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 613 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:37:00.156516 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dd39f5-fa4c-4413-8cef-9c1fbab09ee1 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:00.167431 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f607cde-96c6-4573-b572-45340e203fa9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3e1a2b83f3d4d88ba5084dfaffd22f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:00.168383 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f607cde-96c6-4573-b572-45340e203fa9 service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:00.169480 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f607cde-96c6-4573-b572-45340e203fa9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:00.169920 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f607cde-96c6-4573-b572-45340e203fa9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:00.184964 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f607cde-96c6-4573-b572-45340e203fa9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:00.188977 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f607cde-96c6-4573-b572-45340e203fa9 service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:00.209218 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 887/1760] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:37:00 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 15:37:00.394868 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b286b7-b03f-438f-8510-a7f2d76f805e service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:00.396562 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-23b286b7-b03f-438f-8510-a7f2d76f805e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:00.412846 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:37:00.425035 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b286b7-b03f-438f-8510-a7f2d76f805e service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:00.434267 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca46f12f-d0d7-4331-8e5c-99082f82475a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3e1a2b83f3d4d88ba5084dfaffd22f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:00.434953 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca46f12f-d0d7-4331-8e5c-99082f82475a service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:00.435230 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca46f12f-d0d7-4331-8e5c-99082f82475a service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:00.435516 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca46f12f-d0d7-4331-8e5c-99082f82475a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:00.443380 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ca46f12f-d0d7-4331-8e5c-99082f82475a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:00.675784 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca46f12f-d0d7-4331-8e5c-99082f82475a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:00.677340 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca46f12f-d0d7-4331-8e5c-99082f82475a service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:00.704514 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 874/1761] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:37:00 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:37:01.503860 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f607cde-96c6-4573-b572-45340e203fa9 service cinder] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:01.509168 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-2f607cde-96c6-4573-b572-45340e203fa9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:01.544586 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f607cde-96c6-4573-b572-45340e203fa9 service cinder] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:01.583798 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0befef31-81eb-4802-a209-c9c762dd2d45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:01.599754 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0befef31-81eb-4802-a209-c9c762dd2d45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:01.605009 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0befef31-81eb-4802-a209-c9c762dd2d45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:01.605407 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0befef31-81eb-4802-a209-c9c762dd2d45 admin admin] PATH_INFO: `/v3/projects/0d3d704462bc4886b997f04ba885de7a` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:01.614887 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0befef31-81eb-4802-a209-c9c762dd2d45 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d3d704462bc4886b997f04ba885de7a)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:01.627319 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0befef31-81eb-4802-a209-c9c762dd2d45 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:01.840749 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-0befef31-81eb-4802-a209-c9c762dd2d45 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d3d704462bc4886b997f04ba885de7a', 'request_id': 'req-0befef31-81eb-4802-a209-c9c762dd2d45'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:01.872492 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-0befef31-81eb-4802-a209-c9c762dd2d45 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d3d704462bc4886b997f04ba885de7a', 'request_id': 'req-0befef31-81eb-4802-a209-c9c762dd2d45'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:01.897999 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-0befef31-81eb-4802-a209-c9c762dd2d45 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d3d704462bc4886b997f04ba885de7a', 'request_id': 'req-0befef31-81eb-4802-a209-c9c762dd2d45'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:01.907812 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:37:01.940156 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-0befef31-81eb-4802-a209-c9c762dd2d45 admin admin] The token cache is being invalidate because project 0d3d704462bc4886b997f04ba885de7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 15:37:01.940945 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-0befef31-81eb-4802-a209-c9c762dd2d45 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:01.944680 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-0befef31-81eb-4802-a209-c9c762dd2d45 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:01.949588 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 888/1762] 10.222.0.99 () {58 vars in 1232 bytes} [Thu Jul 14 15:37:01 2022] DELETE /identity/v3/projects/0d3d704462bc4886b997f04ba885de7a => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:37:01.953199 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca46f12f-d0d7-4331-8e5c-99082f82475a service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:01.978518 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca46f12f-d0d7-4331-8e5c-99082f82475a service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:02.132798 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14646830-c10a-4654-8b9a-11f8ccfa1b64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:02.136786 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14646830-c10a-4654-8b9a-11f8ccfa1b64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:02.137192 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14646830-c10a-4654-8b9a-11f8ccfa1b64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:02.137580 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14646830-c10a-4654-8b9a-11f8ccfa1b64 admin admin] PATH_INFO: `/v3/users/7b8e301214be4ffd8c7d39bbddf36f34` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:02.151497 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14646830-c10a-4654-8b9a-11f8ccfa1b64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b8e301214be4ffd8c7d39bbddf36f34)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:02.162702 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14646830-c10a-4654-8b9a-11f8ccfa1b64 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:02.376465 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0befef31-81eb-4802-a209-c9c762dd2d45 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:02.384855 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-0befef31-81eb-4802-a209-c9c762dd2d45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:02.413985 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-14646830-c10a-4654-8b9a-11f8ccfa1b64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b8e301214be4ffd8c7d39bbddf36f34'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:02.430343 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0befef31-81eb-4802-a209-c9c762dd2d45 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:02.439480 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-14646830-c10a-4654-8b9a-11f8ccfa1b64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b8e301214be4ffd8c7d39bbddf36f34'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:02.446054 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d77ee3-9055-464e-8289-985c0dcf6be3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:02.448146 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d77ee3-9055-464e-8289-985c0dcf6be3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:02.448436 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d77ee3-9055-464e-8289-985c0dcf6be3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:02.448764 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d77ee3-9055-464e-8289-985c0dcf6be3 admin admin] PATH_INFO: `/v3/projects/a6c04cbbcc13499585afd5f799cc63aa` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:02.454669 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d77ee3-9055-464e-8289-985c0dcf6be3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6c04cbbcc13499585afd5f799cc63aa)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:02.459762 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d77ee3-9055-464e-8289-985c0dcf6be3 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:02.466610 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-14646830-c10a-4654-8b9a-11f8ccfa1b64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b8e301214be4ffd8c7d39bbddf36f34'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:02.468582 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-14646830-c10a-4654-8b9a-11f8ccfa1b64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b8e301214be4ffd8c7d39bbddf36f34'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:02.469255 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-14646830-c10a-4654-8b9a-11f8ccfa1b64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b8e301214be4ffd8c7d39bbddf36f34'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:02.515042 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 875/1763] 10.222.0.99 () {58 vars in 1223 bytes} [Thu Jul 14 15:37:01 2022] DELETE /identity/v3/users/7b8e301214be4ffd8c7d39bbddf36f34 => generated 0 bytes in 563 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:37:02.585862 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-20d77ee3-9055-464e-8289-985c0dcf6be3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6c04cbbcc13499585afd5f799cc63aa', 'request_id': 'req-20d77ee3-9055-464e-8289-985c0dcf6be3'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:02.596822 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-20d77ee3-9055-464e-8289-985c0dcf6be3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6c04cbbcc13499585afd5f799cc63aa', 'request_id': 'req-20d77ee3-9055-464e-8289-985c0dcf6be3'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:02.604000 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-20d77ee3-9055-464e-8289-985c0dcf6be3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6c04cbbcc13499585afd5f799cc63aa', 'request_id': 'req-20d77ee3-9055-464e-8289-985c0dcf6be3'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:02.647732 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-20d77ee3-9055-464e-8289-985c0dcf6be3 admin admin] The token cache is being invalidate because project a6c04cbbcc13499585afd5f799cc63aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 15:37:02.649248 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-20d77ee3-9055-464e-8289-985c0dcf6be3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:02.650201 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-20d77ee3-9055-464e-8289-985c0dcf6be3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:02.667487 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 889/1764] 10.222.0.99 () {58 vars in 1232 bytes} [Thu Jul 14 15:37:02 2022] DELETE /identity/v3/projects/a6c04cbbcc13499585afd5f799cc63aa => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:37:02.672045 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14646830-c10a-4654-8b9a-11f8ccfa1b64 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:02.673427 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-14646830-c10a-4654-8b9a-11f8ccfa1b64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:02.759566 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:37:02.864383 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14646830-c10a-4654-8b9a-11f8ccfa1b64 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:03.006029 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879d589b-4c39-4698-a4be-efaec270473c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:03.007222 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879d589b-4c39-4698-a4be-efaec270473c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:03.007476 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879d589b-4c39-4698-a4be-efaec270473c admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:03.007792 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879d589b-4c39-4698-a4be-efaec270473c admin admin] PATH_INFO: `/v3/users/a02d99f63ecf4053afb8aa04eccef484` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:03.012052 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879d589b-4c39-4698-a4be-efaec270473c admin admin] RBAC: Authorizing `identity:delete_user(user_id=a02d99f63ecf4053afb8aa04eccef484)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:03.014021 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879d589b-4c39-4698-a4be-efaec270473c admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:03.158313 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-879d589b-4c39-4698-a4be-efaec270473c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a02d99f63ecf4053afb8aa04eccef484'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:03.173713 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-879d589b-4c39-4698-a4be-efaec270473c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a02d99f63ecf4053afb8aa04eccef484'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:03.202659 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-879d589b-4c39-4698-a4be-efaec270473c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a02d99f63ecf4053afb8aa04eccef484'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:03.203470 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-879d589b-4c39-4698-a4be-efaec270473c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a02d99f63ecf4053afb8aa04eccef484'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:03.204278 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-879d589b-4c39-4698-a4be-efaec270473c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a02d99f63ecf4053afb8aa04eccef484'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:03.243346 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 876/1765] 10.222.0.99 () {58 vars in 1223 bytes} [Thu Jul 14 15:37:02 2022] DELETE /identity/v3/users/a02d99f63ecf4053afb8aa04eccef484 => generated 0 bytes in 573 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:37:03.646167 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e92f8e-da4a-47f2-b0fb-222b00288ca8 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:03.646167 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e92f8e-da4a-47f2-b0fb-222b00288ca8 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:03.646749 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e92f8e-da4a-47f2-b0fb-222b00288ca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:03.666405 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-84e92f8e-da4a-47f2-b0fb-222b00288ca8 None None] MFA Rules not processed for user `46f52ad195a34123982d238ea351172d`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:37:03.667995 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-84e92f8e-da4a-47f2-b0fb-222b00288ca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:03.726239 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879d589b-4c39-4698-a4be-efaec270473c admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:03.727968 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-879d589b-4c39-4698-a4be-efaec270473c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:03.755855 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 890/1766] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:37:03 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:37:03.858777 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:37:03.897040 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879d589b-4c39-4698-a4be-efaec270473c admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:03.908762 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb582e10-d93b-4693-a19c-c06cab15459e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:03.910329 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb582e10-d93b-4693-a19c-c06cab15459e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:03.910847 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb582e10-d93b-4693-a19c-c06cab15459e admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:03.911349 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb582e10-d93b-4693-a19c-c06cab15459e admin admin] PATH_INFO: `/v3/projects/599311d17a5e45ecb73ebeaff4393da9` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:03.912968 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb582e10-d93b-4693-a19c-c06cab15459e admin admin] RBAC: Authorizing `identity:delete_project(project_id=599311d17a5e45ecb73ebeaff4393da9)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:03.916867 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb582e10-d93b-4693-a19c-c06cab15459e admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:04.012070 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-cb582e10-d93b-4693-a19c-c06cab15459e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '599311d17a5e45ecb73ebeaff4393da9', 'request_id': 'req-cb582e10-d93b-4693-a19c-c06cab15459e'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:04.024684 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-cb582e10-d93b-4693-a19c-c06cab15459e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '599311d17a5e45ecb73ebeaff4393da9', 'request_id': 'req-cb582e10-d93b-4693-a19c-c06cab15459e'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:04.031530 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-cb582e10-d93b-4693-a19c-c06cab15459e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '599311d17a5e45ecb73ebeaff4393da9', 'request_id': 'req-cb582e10-d93b-4693-a19c-c06cab15459e'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:04.056452 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-cb582e10-d93b-4693-a19c-c06cab15459e admin admin] The token cache is being invalidate because project 599311d17a5e45ecb73ebeaff4393da9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 15:37:04.056874 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-cb582e10-d93b-4693-a19c-c06cab15459e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:04.057603 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-cb582e10-d93b-4693-a19c-c06cab15459e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:04.060434 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 877/1767] 10.222.0.99 () {58 vars in 1232 bytes} [Thu Jul 14 15:37:03 2022] DELETE /identity/v3/projects/599311d17a5e45ecb73ebeaff4393da9 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 15:37:04.066219 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7551c7-9c13-4aa1-ad46-a3716abd0b81 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:04.067577 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7551c7-9c13-4aa1-ad46-a3716abd0b81 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:04.067974 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7551c7-9c13-4aa1-ad46-a3716abd0b81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:04.112870 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-ec7551c7-9c13-4aa1-ad46-a3716abd0b81 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:37:04.127310 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-ec7551c7-9c13-4aa1-ad46-a3716abd0b81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:04.185184 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb582e10-d93b-4693-a19c-c06cab15459e admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:04.187469 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-cb582e10-d93b-4693-a19c-c06cab15459e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:04.313693 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb582e10-d93b-4693-a19c-c06cab15459e admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:04.324489 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 891/1768] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:37:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:37:04.324690 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09f642e-0c27-4d6d-88ff-494095a2dcfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:04.325514 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09f642e-0c27-4d6d-88ff-494095a2dcfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:04.325955 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09f642e-0c27-4d6d-88ff-494095a2dcfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:04.326234 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09f642e-0c27-4d6d-88ff-494095a2dcfe admin admin] PATH_INFO: `/v3/projects/7e8d26e151614d86b9cda47276c6f685` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:04.328047 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09f642e-0c27-4d6d-88ff-494095a2dcfe admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e8d26e151614d86b9cda47276c6f685)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:04.329974 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09f642e-0c27-4d6d-88ff-494095a2dcfe admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:04.333931 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c8f642-9ac9-4edd-9d4e-cd97c127fd4c None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:04.334873 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c8f642-9ac9-4edd-9d4e-cd97c127fd4c None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:04.335383 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c8f642-9ac9-4edd-9d4e-cd97c127fd4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:04.395330 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e09f642e-0c27-4d6d-88ff-494095a2dcfe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e8d26e151614d86b9cda47276c6f685', 'request_id': 'req-e09f642e-0c27-4d6d-88ff-494095a2dcfe'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:04.402623 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e09f642e-0c27-4d6d-88ff-494095a2dcfe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e8d26e151614d86b9cda47276c6f685', 'request_id': 'req-e09f642e-0c27-4d6d-88ff-494095a2dcfe'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:04.410837 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-33c8f642-9ac9-4edd-9d4e-cd97c127fd4c None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:37:04.411060 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e09f642e-0c27-4d6d-88ff-494095a2dcfe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e8d26e151614d86b9cda47276c6f685', 'request_id': 'req-e09f642e-0c27-4d6d-88ff-494095a2dcfe'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:04.412071 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-33c8f642-9ac9-4edd-9d4e-cd97c127fd4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:04.426736 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e09f642e-0c27-4d6d-88ff-494095a2dcfe admin admin] The token cache is being invalidate because project 7e8d26e151614d86b9cda47276c6f685 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 15:37:04.427288 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e09f642e-0c27-4d6d-88ff-494095a2dcfe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:04.428308 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e09f642e-0c27-4d6d-88ff-494095a2dcfe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:04.430553 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 878/1769] 10.222.0.99 () {58 vars in 1232 bytes} [Thu Jul 14 15:37:04 2022] DELETE /identity/v3/projects/7e8d26e151614d86b9cda47276c6f685 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:37:04.436831 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d384ace-78b9-4621-8c56-0b6dfdc2613e None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:04.437451 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d384ace-78b9-4621-8c56-0b6dfdc2613e None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:04.438639 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d384ace-78b9-4621-8c56-0b6dfdc2613e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:04.448699 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:INSERT=2,keystone:DELETE=8,keystone:SELECT=90 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:37:04.478184 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-7d384ace-78b9-4621-8c56-0b6dfdc2613e None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:37:04.479902 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-7d384ace-78b9-4621-8c56-0b6dfdc2613e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:04.487261 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:37:04.528483 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 892/1770] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:37:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:37:04.534830 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c8f642-9ac9-4edd-9d4e-cd97c127fd4c None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:04.588764 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c8f642-9ac9-4edd-9d4e-cd97c127fd4c None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:04.604614 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c73b86-2644-4726-b08e-d1364e90d1e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:04.606081 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 879/1771] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:37:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:37:04.609942 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c73b86-2644-4726-b08e-d1364e90d1e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:04.610312 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c73b86-2644-4726-b08e-d1364e90d1e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:04.611881 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c73b86-2644-4726-b08e-d1364e90d1e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:04.615349 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c73b86-2644-4726-b08e-d1364e90d1e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:37:04.615349 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c73b86-2644-4726-b08e-d1364e90d1e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:04.618715 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c73b86-2644-4726-b08e-d1364e90d1e5 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:04.620681 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff81c51e-7baa-4fca-b716-00ac80bf861a None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:04.620992 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff81c51e-7baa-4fca-b716-00ac80bf861a None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:04.621281 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff81c51e-7baa-4fca-b716-00ac80bf861a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:04.644726 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-ff81c51e-7baa-4fca-b716-00ac80bf861a None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:37:04.646349 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ff81c51e-7baa-4fca-b716-00ac80bf861a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:04.649790 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 893/1772] 10.222.0.99 () {58 vars in 1152 bytes} [Thu Jul 14 15:37:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:37:04.653886 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c73b86-2644-4726-b08e-d1364e90d1e5 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:04.682038 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c73b86-2644-4726-b08e-d1364e90d1e5 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:04.686306 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 880/1773] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:37:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:37:04.690979 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff81c51e-7baa-4fca-b716-00ac80bf861a None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:04.695452 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee37fba-e4aa-4b99-9750-71d40624a902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:04.696171 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee37fba-e4aa-4b99-9750-71d40624a902 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:04.696656 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee37fba-e4aa-4b99-9750-71d40624a902 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:04.696944 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee37fba-e4aa-4b99-9750-71d40624a902 admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:04.697644 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee37fba-e4aa-4b99-9750-71d40624a902 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:04.699777 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee37fba-e4aa-4b99-9750-71d40624a902 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:04.732234 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff81c51e-7baa-4fca-b716-00ac80bf861a None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:04.743363 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e11b32-80a9-4e4e-b59c-78910fd7fa05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:04.744088 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e11b32-80a9-4e4e-b59c-78910fd7fa05 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:04.744381 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e11b32-80a9-4e4e-b59c-78910fd7fa05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:04.744742 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e11b32-80a9-4e4e-b59c-78910fd7fa05 admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:04.745493 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e11b32-80a9-4e4e-b59c-78910fd7fa05 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:37:04.745845 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e11b32-80a9-4e4e-b59c-78910fd7fa05 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:04.752537 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e11b32-80a9-4e4e-b59c-78910fd7fa05 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:04.765429 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 894/1774] 10.222.0.99 () {60 vars in 1152 bytes} [Thu Jul 14 15:37:04 2022] POST /identity/v3/projects => generated 381 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:37:04.769904 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee37fba-e4aa-4b99-9750-71d40624a902 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:04.782409 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 881/1775] 10.222.0.99 () {58 vars in 1152 bytes} [Thu Jul 14 15:37:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:37:04.802735 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e11b32-80a9-4e4e-b59c-78910fd7fa05 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:04.834408 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e11b32-80a9-4e4e-b59c-78910fd7fa05 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:04.883448 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee37fba-e4aa-4b99-9750-71d40624a902 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:04.889243 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adee39c-a90a-4117-8ade-ed77b06de696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:04.889936 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adee39c-a90a-4117-8ade-ed77b06de696 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:04.890223 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adee39c-a90a-4117-8ade-ed77b06de696 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:04.890505 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adee39c-a90a-4117-8ade-ed77b06de696 admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:04.891204 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adee39c-a90a-4117-8ade-ed77b06de696 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:04.893373 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adee39c-a90a-4117-8ade-ed77b06de696 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:04.928628 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 895/1776] 10.222.0.99 () {60 vars in 1143 bytes} [Thu Jul 14 15:37:04 2022] POST /identity/v3/users => generated 291 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:37:04.932693 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adee39c-a90a-4117-8ade-ed77b06de696 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:04.961586 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adee39c-a90a-4117-8ade-ed77b06de696 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:04.977712 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba00f055-d2d1-458e-bc58-86bd7f0832b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:04.981193 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba00f055-d2d1-458e-bc58-86bd7f0832b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:04.981597 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba00f055-d2d1-458e-bc58-86bd7f0832b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:04.981891 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba00f055-d2d1-458e-bc58-86bd7f0832b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:05.000446 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448c2d1b-b95b-4b2f-9054-23680c59392d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:05.001148 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448c2d1b-b95b-4b2f-9054-23680c59392d admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:05.001606 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448c2d1b-b95b-4b2f-9054-23680c59392d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:05.001888 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448c2d1b-b95b-4b2f-9054-23680c59392d admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:05.002722 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448c2d1b-b95b-4b2f-9054-23680c59392d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:05.011185 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448c2d1b-b95b-4b2f-9054-23680c59392d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:05.012565 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba00f055-d2d1-458e-bc58-86bd7f0832b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:05.016224 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba00f055-d2d1-458e-bc58-86bd7f0832b0 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:05.072460 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 896/1777] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:37:04 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:37:05.076120 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba00f055-d2d1-458e-bc58-86bd7f0832b0 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:05.083923 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 882/1778] 10.222.0.99 () {60 vars in 1152 bytes} [Thu Jul 14 15:37:04 2022] POST /identity/v3/projects => generated 369 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:37:05.088117 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448c2d1b-b95b-4b2f-9054-23680c59392d admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:05.096079 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba00f055-d2d1-458e-bc58-86bd7f0832b0 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:05.207785 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad92ed9-037c-441e-8dc6-d7a25ac0151e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:05.207785 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad92ed9-037c-441e-8dc6-d7a25ac0151e admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:05.207785 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad92ed9-037c-441e-8dc6-d7a25ac0151e admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:05.207785 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad92ed9-037c-441e-8dc6-d7a25ac0151e admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:05.208621 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad92ed9-037c-441e-8dc6-d7a25ac0151e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:05.208621 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad92ed9-037c-441e-8dc6-d7a25ac0151e admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:05.213566 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:37:05.226800 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 897/1779] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:37:05 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:37:05.227629 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad92ed9-037c-441e-8dc6-d7a25ac0151e admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:05.244443 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448c2d1b-b95b-4b2f-9054-23680c59392d admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:05.251668 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad92ed9-037c-441e-8dc6-d7a25ac0151e admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:05.257756 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc3e5e7-983b-4edd-ac2d-77316e8cac96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:05.258536 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc3e5e7-983b-4edd-ac2d-77316e8cac96 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:05.258727 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc3e5e7-983b-4edd-ac2d-77316e8cac96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:05.259015 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc3e5e7-983b-4edd-ac2d-77316e8cac96 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:05.260194 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc3e5e7-983b-4edd-ac2d-77316e8cac96 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:05.262547 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233b5978-73f8-41bf-9744-3d9288667548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:05.263376 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233b5978-73f8-41bf-9744-3d9288667548 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:05.263818 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233b5978-73f8-41bf-9744-3d9288667548 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:05.264108 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233b5978-73f8-41bf-9744-3d9288667548 admin admin] PATH_INFO: `/v3/projects/03112c7f482f4bc38dde9bae2e5fe9e3/users/31c8fe96f0e64373b9c3d2b53a8da996/roles/b8c98fddc0fd4436b577fd1d3361bc38` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:05.266611 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc3e5e7-983b-4edd-ac2d-77316e8cac96 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:05.313171 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 883/1780] 10.222.0.99 () {60 vars in 1143 bytes} [Thu Jul 14 15:37:05 2022] POST /identity/v3/users => generated 285 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:37:05.317042 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc3e5e7-983b-4edd-ac2d-77316e8cac96 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:05.325662 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233b5978-73f8-41bf-9744-3d9288667548 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03112c7f482f4bc38dde9bae2e5fe9e3, user_id=31c8fe96f0e64373b9c3d2b53a8da996, role_id=b8c98fddc0fd4436b577fd1d3361bc38)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:05.328516 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:37:05.329315 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233b5978-73f8-41bf-9744-3d9288667548 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:05.333459 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc3e5e7-983b-4edd-ac2d-77316e8cac96 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:05.341717 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a41df03-6456-441f-9a49-d69504b96801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:05.343783 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a41df03-6456-441f-9a49-d69504b96801 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:05.350936 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a41df03-6456-441f-9a49-d69504b96801 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:05.350936 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a41df03-6456-441f-9a49-d69504b96801 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:05.350936 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a41df03-6456-441f-9a49-d69504b96801 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:05.350936 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a41df03-6456-441f-9a49-d69504b96801 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:05.366180 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 898/1781] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 15:37:05 2022] PUT /identity/v3/projects/03112c7f482f4bc38dde9bae2e5fe9e3/users/31c8fe96f0e64373b9c3d2b53a8da996/roles/b8c98fddc0fd4436b577fd1d3361bc38 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:37:05.375659 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371cc608-ec93-4b86-b285-965a21ba8e04 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:05.376150 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371cc608-ec93-4b86-b285-965a21ba8e04 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:05.376790 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371cc608-ec93-4b86-b285-965a21ba8e04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:05.380516 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 884/1782] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:37:05 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:37:05.385477 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a41df03-6456-441f-9a49-d69504b96801 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:05.411108 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a41df03-6456-441f-9a49-d69504b96801 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:05.414095 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-371cc608-ec93-4b86-b285-965a21ba8e04 None None] MFA Rules not processed for user `31c8fe96f0e64373b9c3d2b53a8da996`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:37:05.415484 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-371cc608-ec93-4b86-b285-965a21ba8e04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:05.513960 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 899/1783] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:37:05 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:37:05.520449 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371cc608-ec93-4b86-b285-965a21ba8e04 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:05.534474 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebafded-2b71-47e7-8fdb-d592958ecaa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:05.535252 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebafded-2b71-47e7-8fdb-d592958ecaa3 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:05.535697 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebafded-2b71-47e7-8fdb-d592958ecaa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:05.536133 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebafded-2b71-47e7-8fdb-d592958ecaa3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:05.537060 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebafded-2b71-47e7-8fdb-d592958ecaa3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:05.538847 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebafded-2b71-47e7-8fdb-d592958ecaa3 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:05.564296 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 885/1784] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:37:05 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:37:05.568756 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebafded-2b71-47e7-8fdb-d592958ecaa3 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:05.590407 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebafded-2b71-47e7-8fdb-d592958ecaa3 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:05.603145 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e139c678-190c-4ac2-87b8-645c4b1de59a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:05.604149 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e139c678-190c-4ac2-87b8-645c4b1de59a admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:05.604434 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e139c678-190c-4ac2-87b8-645c4b1de59a admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:05.604717 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e139c678-190c-4ac2-87b8-645c4b1de59a admin admin] PATH_INFO: `/v3/projects/ff57f8c7473b4630ac97eca07d84c2ca/users/2b5feb64c3f14e6487ac3b07b005091d/roles/b8c98fddc0fd4436b577fd1d3361bc38` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:05.641824 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e139c678-190c-4ac2-87b8-645c4b1de59a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff57f8c7473b4630ac97eca07d84c2ca, user_id=2b5feb64c3f14e6487ac3b07b005091d, role_id=b8c98fddc0fd4436b577fd1d3361bc38)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:05.645045 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e139c678-190c-4ac2-87b8-645c4b1de59a admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:05.651063 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371cc608-ec93-4b86-b285-965a21ba8e04 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:05.658779 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa09a11-cfeb-4645-8e1f-55a589d55710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:05.662448 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa09a11-cfeb-4645-8e1f-55a589d55710 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:05.663754 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa09a11-cfeb-4645-8e1f-55a589d55710 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:05.665742 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa09a11-cfeb-4645-8e1f-55a589d55710 admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:05.667980 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa09a11-cfeb-4645-8e1f-55a589d55710 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:05.670254 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa09a11-cfeb-4645-8e1f-55a589d55710 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:05.683681 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 886/1785] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 15:37:05 2022] PUT /identity/v3/projects/ff57f8c7473b4630ac97eca07d84c2ca/users/2b5feb64c3f14e6487ac3b07b005091d/roles/b8c98fddc0fd4436b577fd1d3361bc38 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:37:05.690955 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c202255-1907-4a7c-a3c1-5ecd2fac10b1 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:05.691254 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c202255-1907-4a7c-a3c1-5ecd2fac10b1 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:05.691540 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c202255-1907-4a7c-a3c1-5ecd2fac10b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:05.726892 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 900/1786] 10.222.0.99 () {60 vars in 1152 bytes} [Thu Jul 14 15:37:05 2022] POST /identity/v3/projects => generated 381 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:37:05.728699 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa09a11-cfeb-4645-8e1f-55a589d55710 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:05.731396 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-9c202255-1907-4a7c-a3c1-5ecd2fac10b1 None None] MFA Rules not processed for user `2b5feb64c3f14e6487ac3b07b005091d`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:37:05.732666 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-9c202255-1907-4a7c-a3c1-5ecd2fac10b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:05.812631 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:37:05.825072 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 887/1787] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:37:05 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:37:05.827579 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa09a11-cfeb-4645-8e1f-55a589d55710 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:05.830290 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c202255-1907-4a7c-a3c1-5ecd2fac10b1 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:05.834904 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2529c86e-33bf-47be-bd90-49b9668391e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:05.835367 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2529c86e-33bf-47be-bd90-49b9668391e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:05.835732 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2529c86e-33bf-47be-bd90-49b9668391e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:05.836397 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2529c86e-33bf-47be-bd90-49b9668391e9 admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:05.837522 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2529c86e-33bf-47be-bd90-49b9668391e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:05.839960 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2529c86e-33bf-47be-bd90-49b9668391e9 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:05.851119 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c202255-1907-4a7c-a3c1-5ecd2fac10b1 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:05.859487 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e54fd3-adc6-4307-aa66-9d1099008e96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:05.860424 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e54fd3-adc6-4307-aa66-9d1099008e96 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:05.860814 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e54fd3-adc6-4307-aa66-9d1099008e96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:05.861224 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e54fd3-adc6-4307-aa66-9d1099008e96 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:05.862228 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e54fd3-adc6-4307-aa66-9d1099008e96 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:05.866056 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e54fd3-adc6-4307-aa66-9d1099008e96 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:05.867011 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 901/1788] 10.222.0.99 () {60 vars in 1143 bytes} [Thu Jul 14 15:37:05 2022] POST /identity/v3/users => generated 297 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:37:05.870966 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2529c86e-33bf-47be-bd90-49b9668391e9 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:05.884019 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2529c86e-33bf-47be-bd90-49b9668391e9 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:05.890226 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cf535a-ebef-4dbd-82ca-cd08e4494d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:05.891753 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cf535a-ebef-4dbd-82ca-cd08e4494d2c admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:05.892373 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cf535a-ebef-4dbd-82ca-cd08e4494d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:05.893086 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cf535a-ebef-4dbd-82ca-cd08e4494d2c admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:05.894163 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cf535a-ebef-4dbd-82ca-cd08e4494d2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:05.896405 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cf535a-ebef-4dbd-82ca-cd08e4494d2c admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:05.911981 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 902/1789] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:37:05 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:37:05.912889 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cf535a-ebef-4dbd-82ca-cd08e4494d2c admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:05.917800 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 888/1790] 10.222.0.99 () {60 vars in 1152 bytes} [Thu Jul 14 15:37:05 2022] POST /identity/v3/projects => generated 369 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 15:37:05.919156 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e54fd3-adc6-4307-aa66-9d1099008e96 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:05.931123 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cf535a-ebef-4dbd-82ca-cd08e4494d2c admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:05.983415 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:37:06.019629 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e54fd3-adc6-4307-aa66-9d1099008e96 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:06.024164 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dfd26f-ab49-4184-99a0-618fdab2213f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:06.024979 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08abc360-d2f3-4fb6-9478-b81f8eb3c7fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:06.025111 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dfd26f-ab49-4184-99a0-618fdab2213f admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:06.025398 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dfd26f-ab49-4184-99a0-618fdab2213f admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:06.025664 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dfd26f-ab49-4184-99a0-618fdab2213f admin admin] PATH_INFO: `/v3/projects/ebaf3a1313a74551bf44a8134ea1f792/users/705e38154b7740a2a85a69f138c98388/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:06.026396 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08abc360-d2f3-4fb6-9478-b81f8eb3c7fb admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:06.026834 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08abc360-d2f3-4fb6-9478-b81f8eb3c7fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:06.027257 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08abc360-d2f3-4fb6-9478-b81f8eb3c7fb admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:06.028143 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08abc360-d2f3-4fb6-9478-b81f8eb3c7fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:06.031419 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08abc360-d2f3-4fb6-9478-b81f8eb3c7fb admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:06.047383 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dfd26f-ab49-4184-99a0-618fdab2213f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebaf3a1313a74551bf44a8134ea1f792, user_id=705e38154b7740a2a85a69f138c98388, role_id=d0f6ba1314f14d9f9c816bdb5d0c3aac)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:06.051182 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dfd26f-ab49-4184-99a0-618fdab2213f admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:06.057239 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 889/1791] 10.222.0.99 () {60 vars in 1143 bytes} [Thu Jul 14 15:37:05 2022] POST /identity/v3/users => generated 291 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 15:37:06.061657 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08abc360-d2f3-4fb6-9478-b81f8eb3c7fb admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:06.073974 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 903/1792] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 15:37:05 2022] PUT /identity/v3/projects/ebaf3a1313a74551bf44a8134ea1f792/users/705e38154b7740a2a85a69f138c98388/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:37:06.074163 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08abc360-d2f3-4fb6-9478-b81f8eb3c7fb admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:06.075887 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dfd26f-ab49-4184-99a0-618fdab2213f admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:06.151225 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dfd26f-ab49-4184-99a0-618fdab2213f admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:06.156771 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14c1e35-0965-416d-b3e4-ab0ed39cd0fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:06.157684 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14c1e35-0965-416d-b3e4-ab0ed39cd0fd admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:06.157984 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14c1e35-0965-416d-b3e4-ab0ed39cd0fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:06.158273 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14c1e35-0965-416d-b3e4-ab0ed39cd0fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:06.159182 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14c1e35-0965-416d-b3e4-ab0ed39cd0fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:06.161223 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14c1e35-0965-416d-b3e4-ab0ed39cd0fd admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:06.163513 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6b6e2d-e5fd-493c-b3c1-9fdbb850b7cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:06.164720 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6b6e2d-e5fd-493c-b3c1-9fdbb850b7cc admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:06.165015 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6b6e2d-e5fd-493c-b3c1-9fdbb850b7cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:06.165299 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6b6e2d-e5fd-493c-b3c1-9fdbb850b7cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:06.166809 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6b6e2d-e5fd-493c-b3c1-9fdbb850b7cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:06.169011 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6b6e2d-e5fd-493c-b3c1-9fdbb850b7cc admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:06.172968 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 904/1793] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:37:06 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:37:06.177120 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14c1e35-0965-416d-b3e4-ab0ed39cd0fd admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:06.187931 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 890/1794] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:37:06 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:37:06.188420 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14c1e35-0965-416d-b3e4-ab0ed39cd0fd admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:06.193184 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6b6e2d-e5fd-493c-b3c1-9fdbb850b7cc admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:06.195018 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea780494-124c-45f2-98c0-641041797a29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:06.195963 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea780494-124c-45f2-98c0-641041797a29 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:06.196256 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea780494-124c-45f2-98c0-641041797a29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:06.196540 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea780494-124c-45f2-98c0-641041797a29 admin admin] PATH_INFO: `/v3/projects/ebaf3a1313a74551bf44a8134ea1f792/users/705e38154b7740a2a85a69f138c98388/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:06.198589 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea780494-124c-45f2-98c0-641041797a29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebaf3a1313a74551bf44a8134ea1f792, user_id=705e38154b7740a2a85a69f138c98388, role_id=d0f6ba1314f14d9f9c816bdb5d0c3aac)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:06.201464 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea780494-124c-45f2-98c0-641041797a29 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:06.205094 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6b6e2d-e5fd-493c-b3c1-9fdbb850b7cc admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:06.209636 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51084d57-51e5-4e20-8ed6-7a493f9c67ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:06.210537 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51084d57-51e5-4e20-8ed6-7a493f9c67ec admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:06.210822 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51084d57-51e5-4e20-8ed6-7a493f9c67ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:06.211105 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51084d57-51e5-4e20-8ed6-7a493f9c67ec admin admin] PATH_INFO: `/v3/projects/f79f6563823f4cb2ab4b800486dbad1d/users/68d3fcf34783495d81ec8ec3c58b8d3a/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:06.222474 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 905/1795] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 15:37:06 2022] PUT /identity/v3/projects/ebaf3a1313a74551bf44a8134ea1f792/users/705e38154b7740a2a85a69f138c98388/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 15:37:06.229988 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51084d57-51e5-4e20-8ed6-7a493f9c67ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=f79f6563823f4cb2ab4b800486dbad1d, user_id=68d3fcf34783495d81ec8ec3c58b8d3a, role_id=d0f6ba1314f14d9f9c816bdb5d0c3aac)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:06.232628 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ddce8b-eba9-42b2-a788-f3f866052557 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:06.233072 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ddce8b-eba9-42b2-a788-f3f866052557 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:06.233461 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ddce8b-eba9-42b2-a788-f3f866052557 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:06.234074 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51084d57-51e5-4e20-8ed6-7a493f9c67ec admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:06.263037 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 891/1796] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 15:37:06 2022] PUT /identity/v3/projects/f79f6563823f4cb2ab4b800486dbad1d/users/68d3fcf34783495d81ec8ec3c58b8d3a/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:37:06.268925 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-63ddce8b-eba9-42b2-a788-f3f866052557 None None] MFA Rules not processed for user `705e38154b7740a2a85a69f138c98388`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:37:06.270219 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-63ddce8b-eba9-42b2-a788-f3f866052557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:06.274301 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51084d57-51e5-4e20-8ed6-7a493f9c67ec admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:06.296199 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:37:06.343488 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51084d57-51e5-4e20-8ed6-7a493f9c67ec admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:06.348591 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c59942-761e-41e7-85ab-0076db75599f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:06.349174 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 906/1797] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:37:06 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 15:37:06.349423 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c59942-761e-41e7-85ab-0076db75599f admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:06.349769 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c59942-761e-41e7-85ab-0076db75599f admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:06.350106 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c59942-761e-41e7-85ab-0076db75599f admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:06.350908 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c59942-761e-41e7-85ab-0076db75599f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:06.353407 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c59942-761e-41e7-85ab-0076db75599f admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:06.362951 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ddce8b-eba9-42b2-a788-f3f866052557 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:06.364786 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-63ddce8b-eba9-42b2-a788-f3f866052557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:06.368411 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 892/1798] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:37:06 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 15:37:06.369751 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c59942-761e-41e7-85ab-0076db75599f admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:06.382902 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c59942-761e-41e7-85ab-0076db75599f admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:06.387933 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e517c0-8cf4-462f-bbb7-6093646c61a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:06.388952 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e517c0-8cf4-462f-bbb7-6093646c61a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:06.389255 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e517c0-8cf4-462f-bbb7-6093646c61a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:06.389520 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e517c0-8cf4-462f-bbb7-6093646c61a5 admin admin] PATH_INFO: `/v3/projects/f79f6563823f4cb2ab4b800486dbad1d/users/68d3fcf34783495d81ec8ec3c58b8d3a/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:06.391672 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e517c0-8cf4-462f-bbb7-6093646c61a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f79f6563823f4cb2ab4b800486dbad1d, user_id=68d3fcf34783495d81ec8ec3c58b8d3a, role_id=d0f6ba1314f14d9f9c816bdb5d0c3aac)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:06.394363 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e517c0-8cf4-462f-bbb7-6093646c61a5 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:06.411642 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 893/1799] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 15:37:06 2022] PUT /identity/v3/projects/f79f6563823f4cb2ab4b800486dbad1d/users/68d3fcf34783495d81ec8ec3c58b8d3a/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 15:37:06.419305 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b5b949-e7f3-4020-9022-378982795c4a None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:06.419651 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b5b949-e7f3-4020-9022-378982795c4a None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:06.419949 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b5b949-e7f3-4020-9022-378982795c4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:06.435202 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ddce8b-eba9-42b2-a788-f3f866052557 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:06.452366 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-43b5b949-e7f3-4020-9022-378982795c4a None None] MFA Rules not processed for user `68d3fcf34783495d81ec8ec3c58b8d3a`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:37:06.454064 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-43b5b949-e7f3-4020-9022-378982795c4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:06.461168 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:37:06.506632 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb17a2f-37fb-443c-84ee-7baa73225cee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3e1a2b83f3d4d88ba5084dfaffd22f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:06.507427 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb17a2f-37fb-443c-84ee-7baa73225cee service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:06.507816 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb17a2f-37fb-443c-84ee-7baa73225cee service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:06.508103 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb17a2f-37fb-443c-84ee-7baa73225cee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:06.538531 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 894/1800] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:37:06 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 15:37:06.550215 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b5b949-e7f3-4020-9022-378982795c4a None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:06.551653 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-43b5b949-e7f3-4020-9022-378982795c4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:06.564565 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b5b949-e7f3-4020-9022-378982795c4a None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:06.568165 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7c6f7f-e1f5-47c5-8642-03eb96ebd052 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3e1a2b83f3d4d88ba5084dfaffd22f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:06.568822 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7c6f7f-e1f5-47c5-8642-03eb96ebd052 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:06.569115 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7c6f7f-e1f5-47c5-8642-03eb96ebd052 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:06.569466 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7c6f7f-e1f5-47c5-8642-03eb96ebd052 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:06.573226 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb17a2f-37fb-443c-84ee-7baa73225cee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:06.574716 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb17a2f-37fb-443c-84ee-7baa73225cee service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:06.579761 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7c6f7f-e1f5-47c5-8642-03eb96ebd052 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:06.581576 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7c6f7f-e1f5-47c5-8642-03eb96ebd052 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:06.587208 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 907/1801] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:37:06 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 15:37:06.591906 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 895/1802] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:37:06 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:37:06.637251 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb17a2f-37fb-443c-84ee-7baa73225cee service cinder] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:06.647452 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb17a2f-37fb-443c-84ee-7baa73225cee service cinder] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:06.651375 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758cc6c5-0298-4293-9900-6d1c8032e890 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3e1a2b83f3d4d88ba5084dfaffd22f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:06.652079 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758cc6c5-0298-4293-9900-6d1c8032e890 service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:06.652369 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758cc6c5-0298-4293-9900-6d1c8032e890 service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:06.652963 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758cc6c5-0298-4293-9900-6d1c8032e890 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:06.666176 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7c6f7f-e1f5-47c5-8642-03eb96ebd052 service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:06.675224 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7c6f7f-e1f5-47c5-8642-03eb96ebd052 service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:06.678967 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d87057-f701-4210-a630-33c1bf48d0f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3e1a2b83f3d4d88ba5084dfaffd22f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:06.679616 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d87057-f701-4210-a630-33c1bf48d0f7 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:06.679911 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d87057-f701-4210-a630-33c1bf48d0f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:06.680190 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d87057-f701-4210-a630-33c1bf48d0f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:06.683841 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:37:06.727116 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758cc6c5-0298-4293-9900-6d1c8032e890 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:06.728784 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758cc6c5-0298-4293-9900-6d1c8032e890 service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:06.740182 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 908/1803] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:37:06 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:37:06.755267 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d87057-f701-4210-a630-33c1bf48d0f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:06.756746 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d87057-f701-4210-a630-33c1bf48d0f7 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:06.776798 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 896/1804] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:37:06 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 15:37:07.247545 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758cc6c5-0298-4293-9900-6d1c8032e890 service cinder] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:07.249976 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-758cc6c5-0298-4293-9900-6d1c8032e890 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:07.338491 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758cc6c5-0298-4293-9900-6d1c8032e890 service cinder] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:07.345750 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0157469-f974-47db-b509-0a7dfbbd597a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:07.346841 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0157469-f974-47db-b509-0a7dfbbd597a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:07.347288 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0157469-f974-47db-b509-0a7dfbbd597a admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:07.347708 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0157469-f974-47db-b509-0a7dfbbd597a admin admin] PATH_INFO: `/v3/users/1d641b71e8c54f2188f19e538570a9c2` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:07.349375 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0157469-f974-47db-b509-0a7dfbbd597a admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d641b71e8c54f2188f19e538570a9c2)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:07.351533 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0157469-f974-47db-b509-0a7dfbbd597a admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:07.403205 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-c0157469-f974-47db-b509-0a7dfbbd597a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d641b71e8c54f2188f19e538570a9c2'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:07.412044 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-c0157469-f974-47db-b509-0a7dfbbd597a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d641b71e8c54f2188f19e538570a9c2'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:07.424823 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-c0157469-f974-47db-b509-0a7dfbbd597a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d641b71e8c54f2188f19e538570a9c2'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:07.425528 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-c0157469-f974-47db-b509-0a7dfbbd597a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d641b71e8c54f2188f19e538570a9c2'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:07.426255 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-c0157469-f974-47db-b509-0a7dfbbd597a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d641b71e8c54f2188f19e538570a9c2'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:07.439516 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 909/1805] 10.222.0.99 () {58 vars in 1223 bytes} [Thu Jul 14 15:37:07 2022] DELETE /identity/v3/users/1d641b71e8c54f2188f19e538570a9c2 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:37:07.444049 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9793028-3c9c-4aa2-9cac-45509090cc5b None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:07.444509 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9793028-3c9c-4aa2-9cac-45509090cc5b None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:07.444608 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9793028-3c9c-4aa2-9cac-45509090cc5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:07.462768 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-a9793028-3c9c-4aa2-9cac-45509090cc5b None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:37:07.464055 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a9793028-3c9c-4aa2-9cac-45509090cc5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:07.526514 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 897/1806] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:37:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:37:07.543942 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0157469-f974-47db-b509-0a7dfbbd597a admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:07.545936 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-c0157469-f974-47db-b509-0a7dfbbd597a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:07.597397 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:37:07.611302 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0157469-f974-47db-b509-0a7dfbbd597a admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:07.614991 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a6767d-344b-4b5c-8280-a35af205dfee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:07.615664 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a6767d-344b-4b5c-8280-a35af205dfee service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:07.615957 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a6767d-344b-4b5c-8280-a35af205dfee service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:07.616264 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a6767d-344b-4b5c-8280-a35af205dfee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:07.626222 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a6767d-344b-4b5c-8280-a35af205dfee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:07.627706 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a6767d-344b-4b5c-8280-a35af205dfee service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:07.637509 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 910/1807] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:37:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:37:07.919226 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9793028-3c9c-4aa2-9cac-45509090cc5b None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:07.921331 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a9793028-3c9c-4aa2-9cac-45509090cc5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:07.933196 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9793028-3c9c-4aa2-9cac-45509090cc5b None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:07.938055 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebb8a7d-c696-4f7d-9ab3-45784a77b1cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:07.938922 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebb8a7d-c696-4f7d-9ab3-45784a77b1cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:07.939210 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebb8a7d-c696-4f7d-9ab3-45784a77b1cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:07.939618 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebb8a7d-c696-4f7d-9ab3-45784a77b1cf admin admin] PATH_INFO: `/v3/projects/9411879468964f759dd0ba0d159dfa9f` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:07.940712 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebb8a7d-c696-4f7d-9ab3-45784a77b1cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=9411879468964f759dd0ba0d159dfa9f)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:07.942620 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebb8a7d-c696-4f7d-9ab3-45784a77b1cf admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:07.968988 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:37:07.992956 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-cebb8a7d-c696-4f7d-9ab3-45784a77b1cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9411879468964f759dd0ba0d159dfa9f', 'request_id': 'req-cebb8a7d-c696-4f7d-9ab3-45784a77b1cf'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:08.002177 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-cebb8a7d-c696-4f7d-9ab3-45784a77b1cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9411879468964f759dd0ba0d159dfa9f', 'request_id': 'req-cebb8a7d-c696-4f7d-9ab3-45784a77b1cf'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:08.008684 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-cebb8a7d-c696-4f7d-9ab3-45784a77b1cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9411879468964f759dd0ba0d159dfa9f', 'request_id': 'req-cebb8a7d-c696-4f7d-9ab3-45784a77b1cf'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:08.023585 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-cebb8a7d-c696-4f7d-9ab3-45784a77b1cf admin admin] The token cache is being invalidate because project 9411879468964f759dd0ba0d159dfa9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 15:37:08.023908 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-cebb8a7d-c696-4f7d-9ab3-45784a77b1cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:08.024560 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-cebb8a7d-c696-4f7d-9ab3-45784a77b1cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:08.026169 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 898/1808] 10.222.0.99 () {58 vars in 1232 bytes} [Thu Jul 14 15:37:07 2022] DELETE /identity/v3/projects/9411879468964f759dd0ba0d159dfa9f => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:37:08.032047 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a6767d-344b-4b5c-8280-a35af205dfee service neutron] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:08.040902 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a6767d-344b-4b5c-8280-a35af205dfee service neutron] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:08.097563 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49430653-4c81-4211-9ca9-34ca5f3a4118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:08.098436 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49430653-4c81-4211-9ca9-34ca5f3a4118 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:08.098724 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49430653-4c81-4211-9ca9-34ca5f3a4118 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:08.099007 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49430653-4c81-4211-9ca9-34ca5f3a4118 admin admin] PATH_INFO: `/v3/users/da90d91e159f4d31b7f2506961bb6d5e` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:08.100223 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49430653-4c81-4211-9ca9-34ca5f3a4118 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da90d91e159f4d31b7f2506961bb6d5e)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:08.102157 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49430653-4c81-4211-9ca9-34ca5f3a4118 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:08.151527 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-49430653-4c81-4211-9ca9-34ca5f3a4118 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da90d91e159f4d31b7f2506961bb6d5e'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:08.159843 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-49430653-4c81-4211-9ca9-34ca5f3a4118 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da90d91e159f4d31b7f2506961bb6d5e'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:08.173270 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-49430653-4c81-4211-9ca9-34ca5f3a4118 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da90d91e159f4d31b7f2506961bb6d5e'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:08.174063 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-49430653-4c81-4211-9ca9-34ca5f3a4118 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da90d91e159f4d31b7f2506961bb6d5e'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:08.174813 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-49430653-4c81-4211-9ca9-34ca5f3a4118 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da90d91e159f4d31b7f2506961bb6d5e'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:08.187853 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 911/1809] 10.222.0.99 () {58 vars in 1223 bytes} [Thu Jul 14 15:37:08 2022] DELETE /identity/v3/users/da90d91e159f4d31b7f2506961bb6d5e => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:37:08.560317 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebb8a7d-c696-4f7d-9ab3-45784a77b1cf admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:08.563126 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-cebb8a7d-c696-4f7d-9ab3-45784a77b1cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:08.590985 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebb8a7d-c696-4f7d-9ab3-45784a77b1cf admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:08.682289 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fde923-00a8-410b-8054-dbe1d10b3441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:08.683168 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fde923-00a8-410b-8054-dbe1d10b3441 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:08.683642 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fde923-00a8-410b-8054-dbe1d10b3441 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:08.683935 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fde923-00a8-410b-8054-dbe1d10b3441 admin admin] PATH_INFO: `/v3/projects/76505a2a22534241b3d8f5c8252c7e71` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:08.685745 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fde923-00a8-410b-8054-dbe1d10b3441 admin admin] RBAC: Authorizing `identity:delete_project(project_id=76505a2a22534241b3d8f5c8252c7e71)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:08.687218 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fde923-00a8-410b-8054-dbe1d10b3441 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:08.750468 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-03fde923-00a8-410b-8054-dbe1d10b3441 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76505a2a22534241b3d8f5c8252c7e71', 'request_id': 'req-03fde923-00a8-410b-8054-dbe1d10b3441'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:08.757124 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-03fde923-00a8-410b-8054-dbe1d10b3441 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76505a2a22534241b3d8f5c8252c7e71', 'request_id': 'req-03fde923-00a8-410b-8054-dbe1d10b3441'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:08.763686 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-03fde923-00a8-410b-8054-dbe1d10b3441 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76505a2a22534241b3d8f5c8252c7e71', 'request_id': 'req-03fde923-00a8-410b-8054-dbe1d10b3441'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:08.778082 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-03fde923-00a8-410b-8054-dbe1d10b3441 admin admin] The token cache is being invalidate because project 76505a2a22534241b3d8f5c8252c7e71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 15:37:08.778466 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-03fde923-00a8-410b-8054-dbe1d10b3441 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:08.779185 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-03fde923-00a8-410b-8054-dbe1d10b3441 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:08.782250 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 899/1810] 10.222.0.99 () {58 vars in 1232 bytes} [Thu Jul 14 15:37:08 2022] DELETE /identity/v3/projects/76505a2a22534241b3d8f5c8252c7e71 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:37:08.787093 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42694344-2bda-4cd0-a226-02dcd0a901a8 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:08.787391 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42694344-2bda-4cd0-a226-02dcd0a901a8 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:08.787712 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42694344-2bda-4cd0-a226-02dcd0a901a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:08.806533 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-42694344-2bda-4cd0-a226-02dcd0a901a8 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:37:08.807787 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-42694344-2bda-4cd0-a226-02dcd0a901a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:08.877357 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 912/1811] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:37:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:37:08.887008 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0c1c06-5175-403e-a265-7d7cd230745b None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:08.887300 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0c1c06-5175-403e-a265-7d7cd230745b None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:08.887642 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0c1c06-5175-403e-a265-7d7cd230745b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:08.908975 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-3b0c1c06-5175-403e-a265-7d7cd230745b None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:37:08.910770 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-3b0c1c06-5175-403e-a265-7d7cd230745b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:08.922175 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 900/1812] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:37:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 15:37:08.926623 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42694344-2bda-4cd0-a226-02dcd0a901a8 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:08.937752 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42694344-2bda-4cd0-a226-02dcd0a901a8 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:08.942873 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e091c588-5eb0-4d61-8e66-766f4e6421f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:08.943544 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e091c588-5eb0-4d61-8e66-766f4e6421f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:08.943877 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e091c588-5eb0-4d61-8e66-766f4e6421f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:08.944160 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e091c588-5eb0-4d61-8e66-766f4e6421f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:08.945014 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e091c588-5eb0-4d61-8e66-766f4e6421f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:37:08.945470 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e091c588-5eb0-4d61-8e66-766f4e6421f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:08.947345 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e091c588-5eb0-4d61-8e66-766f4e6421f3 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:08.948987 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:37:08.962728 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 913/1813] 10.222.0.99 () {58 vars in 1152 bytes} [Thu Jul 14 15:37:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:37:08.966707 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0c1c06-5175-403e-a265-7d7cd230745b None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:08.976948 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0c1c06-5175-403e-a265-7d7cd230745b None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:08.981551 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f60cbb-9700-4637-9a45-f59e7347318c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:08.982202 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f60cbb-9700-4637-9a45-f59e7347318c admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:08.982488 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f60cbb-9700-4637-9a45-f59e7347318c admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:08.982770 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f60cbb-9700-4637-9a45-f59e7347318c admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:08.983676 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f60cbb-9700-4637-9a45-f59e7347318c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:08.985698 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f60cbb-9700-4637-9a45-f59e7347318c admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:09.019714 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 901/1814] 10.222.0.99 () {60 vars in 1152 bytes} [Thu Jul 14 15:37:08 2022] POST /identity/v3/projects => generated 385 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:37:09.021075 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e091c588-5eb0-4d61-8e66-766f4e6421f3 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:09.031712 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e091c588-5eb0-4d61-8e66-766f4e6421f3 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:09.094440 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e55125f-8d69-496d-a1ce-fd872e42cc20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:09.095118 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e55125f-8d69-496d-a1ce-fd872e42cc20 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:09.095408 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e55125f-8d69-496d-a1ce-fd872e42cc20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:09.095728 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e55125f-8d69-496d-a1ce-fd872e42cc20 admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:09.096422 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e55125f-8d69-496d-a1ce-fd872e42cc20 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:09.098353 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e55125f-8d69-496d-a1ce-fd872e42cc20 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:09.118958 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 914/1815] 10.222.0.99 () {60 vars in 1143 bytes} [Thu Jul 14 15:37:09 2022] POST /identity/v3/users => generated 293 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:37:09.123175 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f60cbb-9700-4637-9a45-f59e7347318c admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:09.134077 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f60cbb-9700-4637-9a45-f59e7347318c admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:09.138579 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997997fe-806d-4ca1-9688-86c734d245a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:09.139928 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997997fe-806d-4ca1-9688-86c734d245a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:09.140218 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997997fe-806d-4ca1-9688-86c734d245a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:09.140501 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997997fe-806d-4ca1-9688-86c734d245a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:09.141234 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997997fe-806d-4ca1-9688-86c734d245a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:09.142834 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997997fe-806d-4ca1-9688-86c734d245a5 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:09.153624 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 902/1816] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:37:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:37:09.158075 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e55125f-8d69-496d-a1ce-fd872e42cc20 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:09.168027 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e55125f-8d69-496d-a1ce-fd872e42cc20 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:09.172783 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0a6fe8-8f5f-46bd-9fca-0bd47f147de2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:09.173448 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0a6fe8-8f5f-46bd-9fca-0bd47f147de2 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:09.173742 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0a6fe8-8f5f-46bd-9fca-0bd47f147de2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:09.174024 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0a6fe8-8f5f-46bd-9fca-0bd47f147de2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:09.174758 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0a6fe8-8f5f-46bd-9fca-0bd47f147de2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:09.176404 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0a6fe8-8f5f-46bd-9fca-0bd47f147de2 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:09.187036 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 915/1817] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:37:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:37:09.191145 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997997fe-806d-4ca1-9688-86c734d245a5 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:09.196396 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:37:09.203489 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997997fe-806d-4ca1-9688-86c734d245a5 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:09.209484 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7661c15-ae51-48f0-bf49-76ae6074052e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:09.210352 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7661c15-ae51-48f0-bf49-76ae6074052e admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:09.210642 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7661c15-ae51-48f0-bf49-76ae6074052e admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:09.210927 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7661c15-ae51-48f0-bf49-76ae6074052e admin admin] PATH_INFO: `/v3/projects/fa54e2d6fa534891902cd211eb304620/users/681ab39d11344def8ac07be45926f4d4/roles/b8c98fddc0fd4436b577fd1d3361bc38` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:09.226579 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7661c15-ae51-48f0-bf49-76ae6074052e admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa54e2d6fa534891902cd211eb304620, user_id=681ab39d11344def8ac07be45926f4d4, role_id=b8c98fddc0fd4436b577fd1d3361bc38)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:09.229280 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7661c15-ae51-48f0-bf49-76ae6074052e admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:09.244335 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 903/1818] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 15:37:09 2022] PUT /identity/v3/projects/fa54e2d6fa534891902cd211eb304620/users/681ab39d11344def8ac07be45926f4d4/roles/b8c98fddc0fd4436b577fd1d3361bc38 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 15:37:09.256803 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130d3806-076e-4be9-b9e9-952049ab288f None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:09.257107 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130d3806-076e-4be9-b9e9-952049ab288f None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:09.257434 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130d3806-076e-4be9-b9e9-952049ab288f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:09.286467 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-130d3806-076e-4be9-b9e9-952049ab288f None None] MFA Rules not processed for user `681ab39d11344def8ac07be45926f4d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:37:09.287716 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-130d3806-076e-4be9-b9e9-952049ab288f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:09.363674 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 916/1819] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:37:09 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:37:09.367714 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7661c15-ae51-48f0-bf49-76ae6074052e admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:09.432249 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7661c15-ae51-48f0-bf49-76ae6074052e admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:09.437567 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ab2cbd-7709-4e7b-b729-c3f97008d2e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:09.438248 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ab2cbd-7709-4e7b-b729-c3f97008d2e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:09.438540 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ab2cbd-7709-4e7b-b729-c3f97008d2e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:09.438827 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ab2cbd-7709-4e7b-b729-c3f97008d2e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:09.439548 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ab2cbd-7709-4e7b-b729-c3f97008d2e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:09.441494 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ab2cbd-7709-4e7b-b729-c3f97008d2e6 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:09.471162 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 904/1820] 10.222.0.99 () {60 vars in 1152 bytes} [Thu Jul 14 15:37:09 2022] POST /identity/v3/projects => generated 387 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:37:09.475207 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130d3806-076e-4be9-b9e9-952049ab288f None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:09.486635 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130d3806-076e-4be9-b9e9-952049ab288f None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:09.544233 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69904d70-c6de-4017-ab3b-29660ab53691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:09.544909 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69904d70-c6de-4017-ab3b-29660ab53691 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:09.545197 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69904d70-c6de-4017-ab3b-29660ab53691 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:09.545499 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69904d70-c6de-4017-ab3b-29660ab53691 admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:09.546275 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69904d70-c6de-4017-ab3b-29660ab53691 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:09.548463 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69904d70-c6de-4017-ab3b-29660ab53691 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:09.556149 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:37:09.571443 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 917/1821] 10.222.0.99 () {60 vars in 1143 bytes} [Thu Jul 14 15:37:09 2022] POST /identity/v3/users => generated 300 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:37:09.575244 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ab2cbd-7709-4e7b-b729-c3f97008d2e6 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:09.587418 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ab2cbd-7709-4e7b-b729-c3f97008d2e6 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:09.593438 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe21f59-b945-4ffe-a72c-19893a221309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:09.594655 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe21f59-b945-4ffe-a72c-19893a221309 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:09.594981 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe21f59-b945-4ffe-a72c-19893a221309 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:09.595265 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe21f59-b945-4ffe-a72c-19893a221309 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:09.596051 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe21f59-b945-4ffe-a72c-19893a221309 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:09.597680 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe21f59-b945-4ffe-a72c-19893a221309 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:09.618457 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 905/1822] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:37:09 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 15:37:09.623358 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69904d70-c6de-4017-ab3b-29660ab53691 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:09.637248 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69904d70-c6de-4017-ab3b-29660ab53691 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:09.644188 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ac6754-04ef-4378-90c0-223ecac16227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:09.646128 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ac6754-04ef-4378-90c0-223ecac16227 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:09.646418 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ac6754-04ef-4378-90c0-223ecac16227 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:09.646699 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ac6754-04ef-4378-90c0-223ecac16227 admin admin] PATH_INFO: `/v3/projects/617b2be4290b4aa688b2ca54f0d49cf8/users/a4e2936e6c9a4784bf695831903cfd57/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:09.663945 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ac6754-04ef-4378-90c0-223ecac16227 admin admin] RBAC: Authorizing `identity:create_grant(project_id=617b2be4290b4aa688b2ca54f0d49cf8, user_id=a4e2936e6c9a4784bf695831903cfd57, role_id=d0f6ba1314f14d9f9c816bdb5d0c3aac)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:09.666629 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ac6754-04ef-4378-90c0-223ecac16227 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:09.688653 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 918/1823] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 15:37:09 2022] PUT /identity/v3/projects/617b2be4290b4aa688b2ca54f0d49cf8/users/a4e2936e6c9a4784bf695831903cfd57/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:37:09.690022 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe21f59-b945-4ffe-a72c-19893a221309 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:09.700357 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe21f59-b945-4ffe-a72c-19893a221309 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:09.774296 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29baf4c5-daea-434f-acf0-83c2a162c7e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:09.775106 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29baf4c5-daea-434f-acf0-83c2a162c7e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:09.775406 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29baf4c5-daea-434f-acf0-83c2a162c7e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:09.775863 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29baf4c5-daea-434f-acf0-83c2a162c7e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:09.776630 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29baf4c5-daea-434f-acf0-83c2a162c7e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:09.778354 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29baf4c5-daea-434f-acf0-83c2a162c7e9 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:09.793909 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 906/1824] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:37:09 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:37:09.798315 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ac6754-04ef-4378-90c0-223ecac16227 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:09.810195 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ac6754-04ef-4378-90c0-223ecac16227 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:09.816096 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26640e61-0444-446e-a0d7-4f1cc1dbccc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:09.817000 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26640e61-0444-446e-a0d7-4f1cc1dbccc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:09.817492 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26640e61-0444-446e-a0d7-4f1cc1dbccc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:09.818025 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26640e61-0444-446e-a0d7-4f1cc1dbccc4 admin admin] PATH_INFO: `/v3/projects/617b2be4290b4aa688b2ca54f0d49cf8/users/a4e2936e6c9a4784bf695831903cfd57/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:09.820209 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26640e61-0444-446e-a0d7-4f1cc1dbccc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=617b2be4290b4aa688b2ca54f0d49cf8, user_id=a4e2936e6c9a4784bf695831903cfd57, role_id=d0f6ba1314f14d9f9c816bdb5d0c3aac)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:09.822880 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26640e61-0444-446e-a0d7-4f1cc1dbccc4 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:09.847062 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 919/1825] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 15:37:09 2022] PUT /identity/v3/projects/617b2be4290b4aa688b2ca54f0d49cf8/users/a4e2936e6c9a4784bf695831903cfd57/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 15:37:09.848308 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5277ad0b-5eaa-4a16-93ef-1b215401ca44 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:09.848663 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5277ad0b-5eaa-4a16-93ef-1b215401ca44 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:09.848880 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5277ad0b-5eaa-4a16-93ef-1b215401ca44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:09.884780 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-5277ad0b-5eaa-4a16-93ef-1b215401ca44 None None] MFA Rules not processed for user `a4e2936e6c9a4784bf695831903cfd57`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:37:09.887078 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-5277ad0b-5eaa-4a16-93ef-1b215401ca44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:09.939968 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:37:09.986558 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 907/1826] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:37:09 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:37:09.998216 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26640e61-0444-446e-a0d7-4f1cc1dbccc4 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:10.001212 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-26640e61-0444-446e-a0d7-4f1cc1dbccc4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:10.086602 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26640e61-0444-446e-a0d7-4f1cc1dbccc4 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:10.090768 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ce9dfc-f96e-49a3-bbce-4c4f0b7cea19 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3e1a2b83f3d4d88ba5084dfaffd22f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:10.091579 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ce9dfc-f96e-49a3-bbce-4c4f0b7cea19 service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:10.092041 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ce9dfc-f96e-49a3-bbce-4c4f0b7cea19 service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:10.092474 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ce9dfc-f96e-49a3-bbce-4c4f0b7cea19 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:10.108482 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ce9dfc-f96e-49a3-bbce-4c4f0b7cea19 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:10.109941 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ce9dfc-f96e-49a3-bbce-4c4f0b7cea19 service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:10.120090 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 920/1827] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:37:09 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:37:10.478424 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5277ad0b-5eaa-4a16-93ef-1b215401ca44 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:10.479949 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-5277ad0b-5eaa-4a16-93ef-1b215401ca44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:10.596957 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5277ad0b-5eaa-4a16-93ef-1b215401ca44 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:10.605437 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53c5890-4ea1-4413-835c-d3fc32c6ad2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:10.606575 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53c5890-4ea1-4413-835c-d3fc32c6ad2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:10.607068 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53c5890-4ea1-4413-835c-d3fc32c6ad2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:10.607449 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53c5890-4ea1-4413-835c-d3fc32c6ad2c admin admin] PATH_INFO: `/v3/users/2b5feb64c3f14e6487ac3b07b005091d` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:10.609695 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53c5890-4ea1-4413-835c-d3fc32c6ad2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b5feb64c3f14e6487ac3b07b005091d)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:10.612612 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53c5890-4ea1-4413-835c-d3fc32c6ad2c admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:10.745837 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-c53c5890-4ea1-4413-835c-d3fc32c6ad2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b5feb64c3f14e6487ac3b07b005091d'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:10.760533 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-c53c5890-4ea1-4413-835c-d3fc32c6ad2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b5feb64c3f14e6487ac3b07b005091d'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:10.787861 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-c53c5890-4ea1-4413-835c-d3fc32c6ad2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b5feb64c3f14e6487ac3b07b005091d'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:10.790060 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-c53c5890-4ea1-4413-835c-d3fc32c6ad2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b5feb64c3f14e6487ac3b07b005091d'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:10.790842 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-c53c5890-4ea1-4413-835c-d3fc32c6ad2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b5feb64c3f14e6487ac3b07b005091d'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:10.824537 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 908/1828] 10.222.0.99 () {58 vars in 1223 bytes} [Thu Jul 14 15:37:10 2022] DELETE /identity/v3/users/2b5feb64c3f14e6487ac3b07b005091d => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:37:10.827353 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b16d91-e796-4f27-8cac-e8093f7a5425 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:10.827690 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b16d91-e796-4f27-8cac-e8093f7a5425 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:10.828761 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b16d91-e796-4f27-8cac-e8093f7a5425 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:10.862991 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-f3b16d91-e796-4f27-8cac-e8093f7a5425 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:37:10.866374 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-f3b16d91-e796-4f27-8cac-e8093f7a5425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:11.013607 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 921/1829] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:37:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:37:11.028724 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53c5890-4ea1-4413-835c-d3fc32c6ad2c admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:11.031167 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c53c5890-4ea1-4413-835c-d3fc32c6ad2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:11.129414 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53c5890-4ea1-4413-835c-d3fc32c6ad2c admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:11.133544 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aeea06-2917-4e0d-99c5-f67591c682a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:11.134245 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aeea06-2917-4e0d-99c5-f67591c682a0 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:11.134596 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aeea06-2917-4e0d-99c5-f67591c682a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:11.134877 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aeea06-2917-4e0d-99c5-f67591c682a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:11.146446 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:37:11.149988 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1aeea06-2917-4e0d-99c5-f67591c682a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:11.151673 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1aeea06-2917-4e0d-99c5-f67591c682a0 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:11.184913 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 909/1830] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:37:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:37:11.329803 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b16d91-e796-4f27-8cac-e8093f7a5425 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:11.334557 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-f3b16d91-e796-4f27-8cac-e8093f7a5425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:11.363263 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b16d91-e796-4f27-8cac-e8093f7a5425 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:11.380357 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acc9402-e01b-41fe-9c96-8ce98e6bc53b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:11.383327 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acc9402-e01b-41fe-9c96-8ce98e6bc53b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:11.385912 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acc9402-e01b-41fe-9c96-8ce98e6bc53b admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:11.386374 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acc9402-e01b-41fe-9c96-8ce98e6bc53b admin admin] PATH_INFO: `/v3/users/681ab39d11344def8ac07be45926f4d4` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:11.389626 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acc9402-e01b-41fe-9c96-8ce98e6bc53b admin admin] RBAC: Authorizing `identity:delete_user(user_id=681ab39d11344def8ac07be45926f4d4)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:11.394877 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acc9402-e01b-41fe-9c96-8ce98e6bc53b admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:11.488203 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-5acc9402-e01b-41fe-9c96-8ce98e6bc53b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '681ab39d11344def8ac07be45926f4d4'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:11.493583 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:INSERT=5,keystone:SELECT=87,keystone:DELETE=8 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:37:11.503576 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-5acc9402-e01b-41fe-9c96-8ce98e6bc53b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '681ab39d11344def8ac07be45926f4d4'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:11.535149 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-5acc9402-e01b-41fe-9c96-8ce98e6bc53b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '681ab39d11344def8ac07be45926f4d4'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:11.536088 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-5acc9402-e01b-41fe-9c96-8ce98e6bc53b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '681ab39d11344def8ac07be45926f4d4'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:11.540145 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-5acc9402-e01b-41fe-9c96-8ce98e6bc53b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '681ab39d11344def8ac07be45926f4d4'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:11.612817 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 922/1831] 10.222.0.99 () {58 vars in 1223 bytes} [Thu Jul 14 15:37:11 2022] DELETE /identity/v3/users/681ab39d11344def8ac07be45926f4d4 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:37:11.616683 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f276b24-0b26-49f3-af04-8e5bbc9addb0 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:11.617190 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f276b24-0b26-49f3-af04-8e5bbc9addb0 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:11.617284 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f276b24-0b26-49f3-af04-8e5bbc9addb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:11.658743 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-3f276b24-0b26-49f3-af04-8e5bbc9addb0 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:37:11.660578 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-3f276b24-0b26-49f3-af04-8e5bbc9addb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:11.865942 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 910/1832] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:37:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:37:12.112447 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acc9402-e01b-41fe-9c96-8ce98e6bc53b admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:12.114776 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-5acc9402-e01b-41fe-9c96-8ce98e6bc53b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:12.239652 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acc9402-e01b-41fe-9c96-8ce98e6bc53b admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:12.244788 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b847dc1-c7b6-415f-ad7d-45e341370013 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:12.245740 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b847dc1-c7b6-415f-ad7d-45e341370013 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:12.246324 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b847dc1-c7b6-415f-ad7d-45e341370013 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:12.246788 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b847dc1-c7b6-415f-ad7d-45e341370013 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:12.275321 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b847dc1-c7b6-415f-ad7d-45e341370013 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:12.277313 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b847dc1-c7b6-415f-ad7d-45e341370013 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:12.291787 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 923/1833] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:37:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:37:12.648413 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98de185-2279-41bf-8545-faa8cc0bc843 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:12.648861 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98de185-2279-41bf-8545-faa8cc0bc843 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:12.648991 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98de185-2279-41bf-8545-faa8cc0bc843 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:12.650975 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 911/1834] 10.222.0.99 () {54 vars in 921 bytes} [Thu Jul 14 15:37:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:37:12.681879 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01a3b06-1082-498a-8f4e-3a76731188f4 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:12.683462 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01a3b06-1082-498a-8f4e-3a76731188f4 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:12.683921 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01a3b06-1082-498a-8f4e-3a76731188f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:12.752055 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98de185-2279-41bf-8545-faa8cc0bc843 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:12.754036 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b98de185-2279-41bf-8545-faa8cc0bc843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:12.763067 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-f01a3b06-1082-498a-8f4e-3a76731188f4 None None] MFA Rules not processed for user `3a9b1c2622934c5185f3b5f1ea623bc0`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:37:12.771480 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-f01a3b06-1082-498a-8f4e-3a76731188f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:12.800790 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98de185-2279-41bf-8545-faa8cc0bc843 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:12.813390 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af45ba39-c46a-4000-a8a5-9184467f3113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:12.818502 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af45ba39-c46a-4000-a8a5-9184467f3113 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:12.818809 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af45ba39-c46a-4000-a8a5-9184467f3113 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:12.819099 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af45ba39-c46a-4000-a8a5-9184467f3113 admin admin] PATH_INFO: `/v3/projects/ff57f8c7473b4630ac97eca07d84c2ca` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:12.822505 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af45ba39-c46a-4000-a8a5-9184467f3113 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff57f8c7473b4630ac97eca07d84c2ca)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:12.831420 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af45ba39-c46a-4000-a8a5-9184467f3113 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:12.941010 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-af45ba39-c46a-4000-a8a5-9184467f3113 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff57f8c7473b4630ac97eca07d84c2ca', 'request_id': 'req-af45ba39-c46a-4000-a8a5-9184467f3113'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:12.954267 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-af45ba39-c46a-4000-a8a5-9184467f3113 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff57f8c7473b4630ac97eca07d84c2ca', 'request_id': 'req-af45ba39-c46a-4000-a8a5-9184467f3113'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:12.958279 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 924/1835] 10.222.0.99 () {58 vars in 1018 bytes} [Thu Jul 14 15:37:12 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:37:12.964538 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-af45ba39-c46a-4000-a8a5-9184467f3113 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff57f8c7473b4630ac97eca07d84c2ca', 'request_id': 'req-af45ba39-c46a-4000-a8a5-9184467f3113'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:12.980561 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01a3b06-1082-498a-8f4e-3a76731188f4 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:12.982337 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-f01a3b06-1082-498a-8f4e-3a76731188f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:12.993799 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-af45ba39-c46a-4000-a8a5-9184467f3113 admin admin] The token cache is being invalidate because project ff57f8c7473b4630ac97eca07d84c2ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 15:37:12.994107 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-af45ba39-c46a-4000-a8a5-9184467f3113 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:12.994830 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-af45ba39-c46a-4000-a8a5-9184467f3113 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:12.997275 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 912/1836] 10.222.0.99 () {58 vars in 1232 bytes} [Thu Jul 14 15:37:12 2022] DELETE /identity/v3/projects/ff57f8c7473b4630ac97eca07d84c2ca => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 15:37:13.001940 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af45ba39-c46a-4000-a8a5-9184467f3113 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:13.005359 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-af45ba39-c46a-4000-a8a5-9184467f3113 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:13.010417 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01a3b06-1082-498a-8f4e-3a76731188f4 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:13.083805 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:37:13.098536 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c97d96-7851-438e-8108-6995390dc942 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9b1c2622934c5185f3b5f1ea623bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:13.100251 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c97d96-7851-438e-8108-6995390dc942 service nova] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:13.100732 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c97d96-7851-438e-8108-6995390dc942 service nova] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:13.101296 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c97d96-7851-438e-8108-6995390dc942 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:13.103949 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-66c97d96-7851-438e-8108-6995390dc942 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:13.118471 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c97d96-7851-438e-8108-6995390dc942 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:13.120437 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c97d96-7851-438e-8108-6995390dc942 service nova] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:13.134368 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 925/1837] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:37:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 15:37:13.148954 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af45ba39-c46a-4000-a8a5-9184467f3113 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:13.153843 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e9a940-3334-40ec-9311-2e43003445e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:13.155257 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e9a940-3334-40ec-9311-2e43003445e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:13.155614 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e9a940-3334-40ec-9311-2e43003445e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:13.155936 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e9a940-3334-40ec-9311-2e43003445e6 admin admin] PATH_INFO: `/v3/users/68d3fcf34783495d81ec8ec3c58b8d3a` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:13.157503 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e9a940-3334-40ec-9311-2e43003445e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=68d3fcf34783495d81ec8ec3c58b8d3a)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:13.167651 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e9a940-3334-40ec-9311-2e43003445e6 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:13.186447 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:37:13.228876 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-44e9a940-3334-40ec-9311-2e43003445e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68d3fcf34783495d81ec8ec3c58b8d3a'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:13.241212 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-44e9a940-3334-40ec-9311-2e43003445e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68d3fcf34783495d81ec8ec3c58b8d3a'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:13.245902 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c97d96-7851-438e-8108-6995390dc942 service nova] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:13.249564 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-66c97d96-7851-438e-8108-6995390dc942 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:13.277870 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-44e9a940-3334-40ec-9311-2e43003445e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68d3fcf34783495d81ec8ec3c58b8d3a'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:13.278679 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-44e9a940-3334-40ec-9311-2e43003445e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68d3fcf34783495d81ec8ec3c58b8d3a'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:13.279413 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-44e9a940-3334-40ec-9311-2e43003445e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68d3fcf34783495d81ec8ec3c58b8d3a'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:13.298316 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 913/1838] 10.222.0.99 () {58 vars in 1223 bytes} [Thu Jul 14 15:37:13 2022] DELETE /identity/v3/users/68d3fcf34783495d81ec8ec3c58b8d3a => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:37:13.305342 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c97d96-7851-438e-8108-6995390dc942 service nova] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:13.389527 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f20bd01-90e4-4121-9d1b-b134f517b047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:13.391465 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f20bd01-90e4-4121-9d1b-b134f517b047 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:13.391791 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f20bd01-90e4-4121-9d1b-b134f517b047 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:13.392081 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f20bd01-90e4-4121-9d1b-b134f517b047 admin admin] PATH_INFO: `/v3/projects/fa54e2d6fa534891902cd211eb304620` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:13.393573 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f20bd01-90e4-4121-9d1b-b134f517b047 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa54e2d6fa534891902cd211eb304620)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:13.395567 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f20bd01-90e4-4121-9d1b-b134f517b047 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:13.458236 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-1f20bd01-90e4-4121-9d1b-b134f517b047 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa54e2d6fa534891902cd211eb304620', 'request_id': 'req-1f20bd01-90e4-4121-9d1b-b134f517b047'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:13.464709 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-1f20bd01-90e4-4121-9d1b-b134f517b047 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa54e2d6fa534891902cd211eb304620', 'request_id': 'req-1f20bd01-90e4-4121-9d1b-b134f517b047'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:13.473878 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-1f20bd01-90e4-4121-9d1b-b134f517b047 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa54e2d6fa534891902cd211eb304620', 'request_id': 'req-1f20bd01-90e4-4121-9d1b-b134f517b047'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:13.504551 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-1f20bd01-90e4-4121-9d1b-b134f517b047 admin admin] The token cache is being invalidate because project fa54e2d6fa534891902cd211eb304620 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 15:37:13.504883 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-1f20bd01-90e4-4121-9d1b-b134f517b047 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:13.505948 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-1f20bd01-90e4-4121-9d1b-b134f517b047 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:13.507634 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 926/1839] 10.222.0.99 () {58 vars in 1232 bytes} [Thu Jul 14 15:37:13 2022] DELETE /identity/v3/projects/fa54e2d6fa534891902cd211eb304620 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:37:13.512245 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e9a940-3334-40ec-9311-2e43003445e6 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:13.515008 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-44e9a940-3334-40ec-9311-2e43003445e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:13.537599 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e9a940-3334-40ec-9311-2e43003445e6 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:13.607270 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259f4adb-f074-4549-abd8-2cafc54a120f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:13.608770 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259f4adb-f074-4549-abd8-2cafc54a120f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:13.609059 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259f4adb-f074-4549-abd8-2cafc54a120f admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:13.609337 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259f4adb-f074-4549-abd8-2cafc54a120f admin admin] PATH_INFO: `/v3/users/a4e2936e6c9a4784bf695831903cfd57` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:13.610579 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259f4adb-f074-4549-abd8-2cafc54a120f admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4e2936e6c9a4784bf695831903cfd57)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:13.612634 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259f4adb-f074-4549-abd8-2cafc54a120f admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:13.663850 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f20bd01-90e4-4121-9d1b-b134f517b047 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:13.664827 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-259f4adb-f074-4549-abd8-2cafc54a120f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4e2936e6c9a4784bf695831903cfd57'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:13.665431 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-1f20bd01-90e4-4121-9d1b-b134f517b047 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:13.673667 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-259f4adb-f074-4549-abd8-2cafc54a120f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4e2936e6c9a4784bf695831903cfd57'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:13.682194 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f20bd01-90e4-4121-9d1b-b134f517b047 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:13.688441 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990cff7a-c166-4b58-9ad7-cf05d57ae2e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:13.688626 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-259f4adb-f074-4549-abd8-2cafc54a120f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4e2936e6c9a4784bf695831903cfd57'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:13.689327 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990cff7a-c166-4b58-9ad7-cf05d57ae2e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:13.689748 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-259f4adb-f074-4549-abd8-2cafc54a120f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4e2936e6c9a4784bf695831903cfd57'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:13.689969 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990cff7a-c166-4b58-9ad7-cf05d57ae2e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:13.690306 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990cff7a-c166-4b58-9ad7-cf05d57ae2e9 admin admin] PATH_INFO: `/v3/projects/f79f6563823f4cb2ab4b800486dbad1d` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:13.690635 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-259f4adb-f074-4549-abd8-2cafc54a120f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4e2936e6c9a4784bf695831903cfd57'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:13.691838 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990cff7a-c166-4b58-9ad7-cf05d57ae2e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f79f6563823f4cb2ab4b800486dbad1d)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:13.693976 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990cff7a-c166-4b58-9ad7-cf05d57ae2e9 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:13.705580 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 914/1840] 10.222.0.99 () {58 vars in 1223 bytes} [Thu Jul 14 15:37:13 2022] DELETE /identity/v3/users/a4e2936e6c9a4784bf695831903cfd57 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 15:37:13.749854 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-990cff7a-c166-4b58-9ad7-cf05d57ae2e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f79f6563823f4cb2ab4b800486dbad1d', 'request_id': 'req-990cff7a-c166-4b58-9ad7-cf05d57ae2e9'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:13.756374 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-990cff7a-c166-4b58-9ad7-cf05d57ae2e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f79f6563823f4cb2ab4b800486dbad1d', 'request_id': 'req-990cff7a-c166-4b58-9ad7-cf05d57ae2e9'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:13.763422 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-990cff7a-c166-4b58-9ad7-cf05d57ae2e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f79f6563823f4cb2ab4b800486dbad1d', 'request_id': 'req-990cff7a-c166-4b58-9ad7-cf05d57ae2e9'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:13.777488 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-990cff7a-c166-4b58-9ad7-cf05d57ae2e9 admin admin] The token cache is being invalidate because project f79f6563823f4cb2ab4b800486dbad1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 15:37:13.777784 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-990cff7a-c166-4b58-9ad7-cf05d57ae2e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:13.778441 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-990cff7a-c166-4b58-9ad7-cf05d57ae2e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:13.780669 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 927/1841] 10.222.0.99 () {58 vars in 1232 bytes} [Thu Jul 14 15:37:13 2022] DELETE /identity/v3/projects/f79f6563823f4cb2ab4b800486dbad1d => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 15:37:13.786298 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adf67c4-631e-447e-8ede-c520a59dc298 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:13.786617 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adf67c4-631e-447e-8ede-c520a59dc298 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:13.786892 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adf67c4-631e-447e-8ede-c520a59dc298 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:13.806693 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-0adf67c4-631e-447e-8ede-c520a59dc298 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:37:13.807985 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0adf67c4-631e-447e-8ede-c520a59dc298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:13.822669 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:37:13.889143 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 915/1842] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:37:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 15:37:13.893159 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89487724-31dd-45a4-982f-c7d423d28a09 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:13.893459 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89487724-31dd-45a4-982f-c7d423d28a09 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:13.893746 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89487724-31dd-45a4-982f-c7d423d28a09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:13.900379 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:37:13.914859 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-89487724-31dd-45a4-982f-c7d423d28a09 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:37:13.915841 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-89487724-31dd-45a4-982f-c7d423d28a09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:13.930882 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adf67c4-631e-447e-8ede-c520a59dc298 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:13.931752 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 928/1843] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:37:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:37:13.942918 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adf67c4-631e-447e-8ede-c520a59dc298 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:13.951119 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01657356-93e0-4168-b2b2-41e575b94f3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:13.952488 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01657356-93e0-4168-b2b2-41e575b94f3d admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:13.952488 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01657356-93e0-4168-b2b2-41e575b94f3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:13.952693 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01657356-93e0-4168-b2b2-41e575b94f3d admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:13.953909 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01657356-93e0-4168-b2b2-41e575b94f3d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:37:13.953909 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01657356-93e0-4168-b2b2-41e575b94f3d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:13.955755 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01657356-93e0-4168-b2b2-41e575b94f3d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:13.977005 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 916/1844] 10.222.0.99 () {58 vars in 1152 bytes} [Thu Jul 14 15:37:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:37:13.978132 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89487724-31dd-45a4-982f-c7d423d28a09 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:13.989831 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89487724-31dd-45a4-982f-c7d423d28a09 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:13.994955 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9043a04b-7aed-4b93-8999-f6ccd0843ecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:13.995663 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9043a04b-7aed-4b93-8999-f6ccd0843ecd admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:13.995956 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9043a04b-7aed-4b93-8999-f6ccd0843ecd admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:13.996235 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9043a04b-7aed-4b93-8999-f6ccd0843ecd admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:13.996963 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9043a04b-7aed-4b93-8999-f6ccd0843ecd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:13.999295 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9043a04b-7aed-4b93-8999-f6ccd0843ecd admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:14.023662 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01657356-93e0-4168-b2b2-41e575b94f3d admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:14.025407 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-01657356-93e0-4168-b2b2-41e575b94f3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:14.032546 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 929/1845] 10.222.0.99 () {60 vars in 1152 bytes} [Thu Jul 14 15:37:13 2022] POST /identity/v3/projects => generated 407 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 15:37:14.036671 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9043a04b-7aed-4b93-8999-f6ccd0843ecd admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:14.042231 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01657356-93e0-4168-b2b2-41e575b94f3d admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:14.111810 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9043a04b-7aed-4b93-8999-f6ccd0843ecd admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:14.114304 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67155234-9100-4d93-9e95-c033111e02ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:14.115255 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67155234-9100-4d93-9e95-c033111e02ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:14.115550 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67155234-9100-4d93-9e95-c033111e02ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:14.115859 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67155234-9100-4d93-9e95-c033111e02ab admin admin] PATH_INFO: `/v3/projects/617b2be4290b4aa688b2ca54f0d49cf8` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:14.117066 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67155234-9100-4d93-9e95-c033111e02ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=617b2be4290b4aa688b2ca54f0d49cf8)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:14.117321 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b5559e-de3d-4a55-8a04-da7cf251acba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:14.118181 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b5559e-de3d-4a55-8a04-da7cf251acba admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:14.118583 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b5559e-de3d-4a55-8a04-da7cf251acba admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:14.118871 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b5559e-de3d-4a55-8a04-da7cf251acba admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:14.119238 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67155234-9100-4d93-9e95-c033111e02ab admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:14.119575 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b5559e-de3d-4a55-8a04-da7cf251acba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:14.122096 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b5559e-de3d-4a55-8a04-da7cf251acba admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:14.148110 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 930/1846] 10.222.0.99 () {60 vars in 1143 bytes} [Thu Jul 14 15:37:14 2022] POST /identity/v3/users => generated 304 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 15:37:14.149184 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b5559e-de3d-4a55-8a04-da7cf251acba admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:14.160967 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b5559e-de3d-4a55-8a04-da7cf251acba admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:14.178632 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-67155234-9100-4d93-9e95-c033111e02ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '617b2be4290b4aa688b2ca54f0d49cf8', 'request_id': 'req-67155234-9100-4d93-9e95-c033111e02ab'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:14.186940 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-67155234-9100-4d93-9e95-c033111e02ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '617b2be4290b4aa688b2ca54f0d49cf8', 'request_id': 'req-67155234-9100-4d93-9e95-c033111e02ab'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:14.193403 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-67155234-9100-4d93-9e95-c033111e02ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '617b2be4290b4aa688b2ca54f0d49cf8', 'request_id': 'req-67155234-9100-4d93-9e95-c033111e02ab'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:14.208440 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-67155234-9100-4d93-9e95-c033111e02ab admin admin] The token cache is being invalidate because project 617b2be4290b4aa688b2ca54f0d49cf8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 15:37:14.208737 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-67155234-9100-4d93-9e95-c033111e02ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:14.209635 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-67155234-9100-4d93-9e95-c033111e02ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:14.211642 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 917/1847] 10.222.0.99 () {58 vars in 1232 bytes} [Thu Jul 14 15:37:14 2022] DELETE /identity/v3/projects/617b2be4290b4aa688b2ca54f0d49cf8 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 15:37:14.216794 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff4497b-345f-4a60-b2ad-162cd5696811 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:14.217141 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff4497b-345f-4a60-b2ad-162cd5696811 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:14.218132 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff4497b-345f-4a60-b2ad-162cd5696811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:14.231573 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27f4e09-aacb-4392-b176-0c6337cf20fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:14.232295 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27f4e09-aacb-4392-b176-0c6337cf20fb admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:14.233234 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27f4e09-aacb-4392-b176-0c6337cf20fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:14.233521 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27f4e09-aacb-4392-b176-0c6337cf20fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:14.234270 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27f4e09-aacb-4392-b176-0c6337cf20fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:14.235926 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27f4e09-aacb-4392-b176-0c6337cf20fb admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:14.244109 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-2ff4497b-345f-4a60-b2ad-162cd5696811 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:37:14.245326 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-2ff4497b-345f-4a60-b2ad-162cd5696811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:14.246720 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 931/1848] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:37:14 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:37:14.251853 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27f4e09-aacb-4392-b176-0c6337cf20fb admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:14.253566 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-e27f4e09-aacb-4392-b176-0c6337cf20fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:14.257829 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 918/1849] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:37:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 15:37:14.266825 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7161931-e95e-4668-88a7-21dc173d940d None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:14.267127 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7161931-e95e-4668-88a7-21dc173d940d None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:14.267413 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7161931-e95e-4668-88a7-21dc173d940d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:14.270613 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27f4e09-aacb-4392-b176-0c6337cf20fb admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:14.276080 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a794b506-1255-4126-8468-11f546632197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:14.276763 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a794b506-1255-4126-8468-11f546632197 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:14.277053 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a794b506-1255-4126-8468-11f546632197 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:14.277339 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a794b506-1255-4126-8468-11f546632197 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:14.279302 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a794b506-1255-4126-8468-11f546632197 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:14.280834 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a794b506-1255-4126-8468-11f546632197 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:14.289370 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-b7161931-e95e-4668-88a7-21dc173d940d None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:37:14.290667 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b7161931-e95e-4668-88a7-21dc173d940d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:14.291502 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 932/1850] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:37:14 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 15:37:14.296550 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a794b506-1255-4126-8468-11f546632197 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:14.304058 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 919/1851] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:37:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:37:14.306682 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a794b506-1255-4126-8468-11f546632197 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:14.308036 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7161931-e95e-4668-88a7-21dc173d940d None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:14.312231 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff13116-9718-41e4-b648-4b25fb27165e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:14.313392 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff13116-9718-41e4-b648-4b25fb27165e admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:14.313689 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff13116-9718-41e4-b648-4b25fb27165e admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:14.313977 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff13116-9718-41e4-b648-4b25fb27165e admin admin] PATH_INFO: `/v3/projects/8b724f3c688144a09418b78827eebe65/users/8f8b17036d4d4f04976f9a203258347b/roles/b8c98fddc0fd4436b577fd1d3361bc38` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:14.318455 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7161931-e95e-4668-88a7-21dc173d940d None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:14.325500 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6981d2d6-69ad-4843-85ab-682ce58d09d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:14.326286 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6981d2d6-69ad-4843-85ab-682ce58d09d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:14.332140 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6981d2d6-69ad-4843-85ab-682ce58d09d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:14.332837 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6981d2d6-69ad-4843-85ab-682ce58d09d9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:14.333947 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6981d2d6-69ad-4843-85ab-682ce58d09d9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:37:14.334403 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6981d2d6-69ad-4843-85ab-682ce58d09d9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:14.334923 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff13116-9718-41e4-b648-4b25fb27165e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b724f3c688144a09418b78827eebe65, user_id=8f8b17036d4d4f04976f9a203258347b, role_id=b8c98fddc0fd4436b577fd1d3361bc38)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:14.336522 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6981d2d6-69ad-4843-85ab-682ce58d09d9 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:14.338268 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff13116-9718-41e4-b648-4b25fb27165e admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:14.355647 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 920/1852] 10.222.0.99 () {58 vars in 1152 bytes} [Thu Jul 14 15:37:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:37:14.360091 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6981d2d6-69ad-4843-85ab-682ce58d09d9 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:14.367837 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 933/1853] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 15:37:14 2022] PUT /identity/v3/projects/8b724f3c688144a09418b78827eebe65/users/8f8b17036d4d4f04976f9a203258347b/roles/b8c98fddc0fd4436b577fd1d3361bc38 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 15:37:14.370242 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6981d2d6-69ad-4843-85ab-682ce58d09d9 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:14.376653 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:37:14.379838 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c50c383-e83f-46f9-b158-8615aff0eebe None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:14.380153 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c50c383-e83f-46f9-b158-8615aff0eebe None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:14.380460 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c50c383-e83f-46f9-b158-8615aff0eebe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:14.408279 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-2c50c383-e83f-46f9-b158-8615aff0eebe None None] MFA Rules not processed for user `8f8b17036d4d4f04976f9a203258347b`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:37:14.409574 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-2c50c383-e83f-46f9-b158-8615aff0eebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:14.421886 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:37:14.434591 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97b88a5-fabc-4724-9351-d81b64bde6fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:14.435271 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97b88a5-fabc-4724-9351-d81b64bde6fd admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:14.435584 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97b88a5-fabc-4724-9351-d81b64bde6fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:14.436448 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97b88a5-fabc-4724-9351-d81b64bde6fd admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:14.437133 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97b88a5-fabc-4724-9351-d81b64bde6fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:14.439064 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97b88a5-fabc-4724-9351-d81b64bde6fd admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:14.471847 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 921/1854] 10.222.0.99 () {60 vars in 1152 bytes} [Thu Jul 14 15:37:14 2022] POST /identity/v3/projects => generated 373 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 15:37:14.475746 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97b88a5-fabc-4724-9351-d81b64bde6fd admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:14.486350 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 934/1855] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:37:14 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 15:37:14.490941 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c50c383-e83f-46f9-b158-8615aff0eebe None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:14.507448 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c50c383-e83f-46f9-b158-8615aff0eebe None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:14.553957 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97b88a5-fabc-4724-9351-d81b64bde6fd admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:14.560220 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e1607c-7574-4028-93b5-7a3e553db6ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:14.561522 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e1607c-7574-4028-93b5-7a3e553db6ee admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:14.561813 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e1607c-7574-4028-93b5-7a3e553db6ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:14.562090 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e1607c-7574-4028-93b5-7a3e553db6ee admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:14.562838 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e1607c-7574-4028-93b5-7a3e553db6ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:14.564925 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e1607c-7574-4028-93b5-7a3e553db6ee admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:14.584920 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727b2465-4e7c-4f7c-b4bd-901ad2e0aef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:14.586384 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727b2465-4e7c-4f7c-b4bd-901ad2e0aef2 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:14.586384 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727b2465-4e7c-4f7c-b4bd-901ad2e0aef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:14.586384 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727b2465-4e7c-4f7c-b4bd-901ad2e0aef2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:14.587025 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727b2465-4e7c-4f7c-b4bd-901ad2e0aef2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:14.589055 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727b2465-4e7c-4f7c-b4bd-901ad2e0aef2 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:14.593107 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 922/1856] 10.222.0.99 () {60 vars in 1143 bytes} [Thu Jul 14 15:37:14 2022] POST /identity/v3/users => generated 287 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 15:37:14.602740 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e1607c-7574-4028-93b5-7a3e553db6ee admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:14.613407 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e1607c-7574-4028-93b5-7a3e553db6ee admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:14.620119 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785f85ce-813a-419c-966a-d5f30ea079d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:14.620786 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785f85ce-813a-419c-966a-d5f30ea079d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:14.621070 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785f85ce-813a-419c-966a-d5f30ea079d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:14.621347 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785f85ce-813a-419c-966a-d5f30ea079d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:14.623761 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 935/1857] 10.222.0.99 () {60 vars in 1152 bytes} [Thu Jul 14 15:37:14 2022] POST /identity/v3/projects => generated 409 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 15:37:14.623897 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785f85ce-813a-419c-966a-d5f30ea079d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:14.624450 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785f85ce-813a-419c-966a-d5f30ea079d3 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:14.626616 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727b2465-4e7c-4f7c-b4bd-901ad2e0aef2 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:14.638239 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 923/1858] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:37:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:37:14.639944 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785f85ce-813a-419c-966a-d5f30ea079d3 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:14.649955 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785f85ce-813a-419c-966a-d5f30ea079d3 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:14.692305 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727b2465-4e7c-4f7c-b4bd-901ad2e0aef2 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:14.697220 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a918df-d106-4c0e-9aa9-27ac78966dcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:14.697929 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a918df-d106-4c0e-9aa9-27ac78966dcc admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:14.698217 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a918df-d106-4c0e-9aa9-27ac78966dcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:14.698500 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a918df-d106-4c0e-9aa9-27ac78966dcc admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:14.699187 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a918df-d106-4c0e-9aa9-27ac78966dcc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:14.701158 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a918df-d106-4c0e-9aa9-27ac78966dcc admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:14.709044 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9b70b4-e4a1-4ce4-945f-9a5012667e7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:14.709727 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9b70b4-e4a1-4ce4-945f-9a5012667e7f admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:14.710148 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9b70b4-e4a1-4ce4-945f-9a5012667e7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:14.710307 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9b70b4-e4a1-4ce4-945f-9a5012667e7f admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:14.711098 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9b70b4-e4a1-4ce4-945f-9a5012667e7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:14.712900 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9b70b4-e4a1-4ce4-945f-9a5012667e7f admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:14.722764 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 936/1859] 10.222.0.99 () {60 vars in 1143 bytes} [Thu Jul 14 15:37:14 2022] POST /identity/v3/users => generated 311 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 15:37:14.726922 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 924/1860] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:37:14 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:37:14.727039 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a918df-d106-4c0e-9aa9-27ac78966dcc admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:14.727866 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9b70b4-e4a1-4ce4-945f-9a5012667e7f admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:14.738842 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a918df-d106-4c0e-9aa9-27ac78966dcc admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:14.740040 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9b70b4-e4a1-4ce4-945f-9a5012667e7f admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:14.743746 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de65bd62-46de-4700-9dc3-858c82d1534f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:14.744421 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de65bd62-46de-4700-9dc3-858c82d1534f admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:14.744677 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10853d1-5ad3-4bd3-a0fa-4b0d106c3c7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:14.744786 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de65bd62-46de-4700-9dc3-858c82d1534f admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:14.744983 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de65bd62-46de-4700-9dc3-858c82d1534f admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:14.745551 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10853d1-5ad3-4bd3-a0fa-4b0d106c3c7d admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:14.745720 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de65bd62-46de-4700-9dc3-858c82d1534f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:14.745820 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10853d1-5ad3-4bd3-a0fa-4b0d106c3c7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:14.746106 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10853d1-5ad3-4bd3-a0fa-4b0d106c3c7d admin admin] PATH_INFO: `/v3/projects/84580006f8ec446b8a5db16ce3e7db15/users/0ca38419e1ad47bfa9b6442bcfec11f0/roles/b8c98fddc0fd4436b577fd1d3361bc38` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:14.747327 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de65bd62-46de-4700-9dc3-858c82d1534f admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:14.750096 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:37:14.759047 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 937/1861] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:37:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:37:14.763240 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de65bd62-46de-4700-9dc3-858c82d1534f admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:14.763694 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10853d1-5ad3-4bd3-a0fa-4b0d106c3c7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=84580006f8ec446b8a5db16ce3e7db15, user_id=0ca38419e1ad47bfa9b6442bcfec11f0, role_id=b8c98fddc0fd4436b577fd1d3361bc38)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:14.767058 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10853d1-5ad3-4bd3-a0fa-4b0d106c3c7d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:14.774744 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de65bd62-46de-4700-9dc3-858c82d1534f admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:14.779416 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8327ebb-381e-4b68-8dd1-5fe365a2ecf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:14.780998 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8327ebb-381e-4b68-8dd1-5fe365a2ecf3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:14.781288 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8327ebb-381e-4b68-8dd1-5fe365a2ecf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:14.781564 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8327ebb-381e-4b68-8dd1-5fe365a2ecf3 admin admin] PATH_INFO: `/v3/projects/e3fb9d7add7544b0b07be3e2598eab4c/users/8cb60118608742c1bfe3c0f93a9d99b9/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:14.782956 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 925/1862] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 15:37:14 2022] PUT /identity/v3/projects/84580006f8ec446b8a5db16ce3e7db15/users/0ca38419e1ad47bfa9b6442bcfec11f0/roles/b8c98fddc0fd4436b577fd1d3361bc38 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 15:37:14.793790 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eec93a-85b2-42bc-a2e7-1d26bb798b66 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:14.794082 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eec93a-85b2-42bc-a2e7-1d26bb798b66 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:14.794361 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eec93a-85b2-42bc-a2e7-1d26bb798b66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:14.806694 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8327ebb-381e-4b68-8dd1-5fe365a2ecf3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3fb9d7add7544b0b07be3e2598eab4c, user_id=8cb60118608742c1bfe3c0f93a9d99b9, role_id=d0f6ba1314f14d9f9c816bdb5d0c3aac)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:14.809412 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8327ebb-381e-4b68-8dd1-5fe365a2ecf3 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:14.811864 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:37:14.822593 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-60eec93a-85b2-42bc-a2e7-1d26bb798b66 None None] MFA Rules not processed for user `0ca38419e1ad47bfa9b6442bcfec11f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:37:14.823858 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-60eec93a-85b2-42bc-a2e7-1d26bb798b66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:14.824333 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 938/1863] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 15:37:14 2022] PUT /identity/v3/projects/e3fb9d7add7544b0b07be3e2598eab4c/users/8cb60118608742c1bfe3c0f93a9d99b9/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 15:37:14.829311 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8327ebb-381e-4b68-8dd1-5fe365a2ecf3 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:14.891892 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 926/1864] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:37:14 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 15:37:14.893429 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8327ebb-381e-4b68-8dd1-5fe365a2ecf3 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:14.898571 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5355409c-8ca4-45bd-9eaf-7009d2c7268a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:14.899249 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5355409c-8ca4-45bd-9eaf-7009d2c7268a admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:14.899540 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5355409c-8ca4-45bd-9eaf-7009d2c7268a admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:14.899853 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5355409c-8ca4-45bd-9eaf-7009d2c7268a admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:14.900587 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5355409c-8ca4-45bd-9eaf-7009d2c7268a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:14.902201 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5355409c-8ca4-45bd-9eaf-7009d2c7268a admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:14.903547 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eec93a-85b2-42bc-a2e7-1d26bb798b66 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:14.905440 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-60eec93a-85b2-42bc-a2e7-1d26bb798b66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:14.916994 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 939/1865] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:37:14 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:37:14.918236 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5355409c-8ca4-45bd-9eaf-7009d2c7268a admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:14.928346 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5355409c-8ca4-45bd-9eaf-7009d2c7268a admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:14.933053 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da660bd-5326-47f1-826b-38dbd02a9ff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:14.933939 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da660bd-5326-47f1-826b-38dbd02a9ff0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:14.934259 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da660bd-5326-47f1-826b-38dbd02a9ff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:14.934545 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da660bd-5326-47f1-826b-38dbd02a9ff0 admin admin] PATH_INFO: `/v3/projects/e3fb9d7add7544b0b07be3e2598eab4c/users/8cb60118608742c1bfe3c0f93a9d99b9/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:14.936618 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da660bd-5326-47f1-826b-38dbd02a9ff0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3fb9d7add7544b0b07be3e2598eab4c, user_id=8cb60118608742c1bfe3c0f93a9d99b9, role_id=d0f6ba1314f14d9f9c816bdb5d0c3aac)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:14.939267 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da660bd-5326-47f1-826b-38dbd02a9ff0 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:14.950405 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 940/1866] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 15:37:14 2022] PUT /identity/v3/projects/e3fb9d7add7544b0b07be3e2598eab4c/users/8cb60118608742c1bfe3c0f93a9d99b9/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 15:37:14.961965 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596f7aac-9970-483f-85e5-68c42f824aa5 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:14.962341 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596f7aac-9970-483f-85e5-68c42f824aa5 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:14.962625 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596f7aac-9970-483f-85e5-68c42f824aa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:14.964622 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eec93a-85b2-42bc-a2e7-1d26bb798b66 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:14.991376 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-596f7aac-9970-483f-85e5-68c42f824aa5 None None] MFA Rules not processed for user `8cb60118608742c1bfe3c0f93a9d99b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:37:14.993157 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-596f7aac-9970-483f-85e5-68c42f824aa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:15.021326 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08104437-d3ea-4610-9054-6e39a012a614 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3e1a2b83f3d4d88ba5084dfaffd22f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:15.021919 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08104437-d3ea-4610-9054-6e39a012a614 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:15.022248 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08104437-d3ea-4610-9054-6e39a012a614 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:15.022533 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08104437-d3ea-4610-9054-6e39a012a614 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:15.079352 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 941/1867] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:37:14 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 15:37:15.079809 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:37:15.091534 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596f7aac-9970-483f-85e5-68c42f824aa5 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:15.093879 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-596f7aac-9970-483f-85e5-68c42f824aa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:15.100514 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08104437-d3ea-4610-9054-6e39a012a614 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:15.102422 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08104437-d3ea-4610-9054-6e39a012a614 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:15.106482 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596f7aac-9970-483f-85e5-68c42f824aa5 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:15.110564 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51ff4cc-2a6e-4b74-b170-6b4cefbc09c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3e1a2b83f3d4d88ba5084dfaffd22f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:15.111380 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51ff4cc-2a6e-4b74-b170-6b4cefbc09c6 service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:15.111725 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51ff4cc-2a6e-4b74-b170-6b4cefbc09c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:15.112015 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51ff4cc-2a6e-4b74-b170-6b4cefbc09c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:15.112808 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 927/1868] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:37:14 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 15:37:15.134997 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08104437-d3ea-4610-9054-6e39a012a614 service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:15.135208 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51ff4cc-2a6e-4b74-b170-6b4cefbc09c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:15.137016 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51ff4cc-2a6e-4b74-b170-6b4cefbc09c6 service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:15.149569 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 942/1869] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:37:15 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 15:37:15.206292 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08104437-d3ea-4610-9054-6e39a012a614 service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:15.211357 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9082995-06fa-49f7-ae84-486fcfa18174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:15.212267 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9082995-06fa-49f7-ae84-486fcfa18174 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:15.212554 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9082995-06fa-49f7-ae84-486fcfa18174 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:15.212835 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9082995-06fa-49f7-ae84-486fcfa18174 admin admin] PATH_INFO: `/v3/users/0ca38419e1ad47bfa9b6442bcfec11f0` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:15.214065 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9082995-06fa-49f7-ae84-486fcfa18174 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ca38419e1ad47bfa9b6442bcfec11f0)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:15.216042 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9082995-06fa-49f7-ae84-486fcfa18174 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:15.263459 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-b9082995-06fa-49f7-ae84-486fcfa18174 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ca38419e1ad47bfa9b6442bcfec11f0'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:15.274164 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-b9082995-06fa-49f7-ae84-486fcfa18174 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ca38419e1ad47bfa9b6442bcfec11f0'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:15.289031 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-b9082995-06fa-49f7-ae84-486fcfa18174 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ca38419e1ad47bfa9b6442bcfec11f0'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:15.290117 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-b9082995-06fa-49f7-ae84-486fcfa18174 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ca38419e1ad47bfa9b6442bcfec11f0'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:15.290839 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-b9082995-06fa-49f7-ae84-486fcfa18174 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ca38419e1ad47bfa9b6442bcfec11f0'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:15.304910 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 928/1870] 10.222.0.99 () {58 vars in 1223 bytes} [Thu Jul 14 15:37:15 2022] DELETE /identity/v3/users/0ca38419e1ad47bfa9b6442bcfec11f0 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 15:37:15.309309 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdb7ef3-e45c-44a7-adee-a7ddc25ac8dc None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:15.309727 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdb7ef3-e45c-44a7-adee-a7ddc25ac8dc None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:15.310096 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdb7ef3-e45c-44a7-adee-a7ddc25ac8dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:15.337031 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-0cdb7ef3-e45c-44a7-adee-a7ddc25ac8dc None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:37:15.338311 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-0cdb7ef3-e45c-44a7-adee-a7ddc25ac8dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:15.342416 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9082995-06fa-49f7-ae84-486fcfa18174 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:15.343971 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b9082995-06fa-49f7-ae84-486fcfa18174 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:15.391494 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:37:15.414704 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 943/1871] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:37:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 15:37:15.426854 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdb7ef3-e45c-44a7-adee-a7ddc25ac8dc None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:15.430096 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-0cdb7ef3-e45c-44a7-adee-a7ddc25ac8dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:15.445462 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9082995-06fa-49f7-ae84-486fcfa18174 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:15.449230 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42f4353-efe7-44eb-93d9-86882ae4b532 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3e1a2b83f3d4d88ba5084dfaffd22f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:15.450045 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42f4353-efe7-44eb-93d9-86882ae4b532 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:15.450336 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42f4353-efe7-44eb-93d9-86882ae4b532 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:15.450788 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42f4353-efe7-44eb-93d9-86882ae4b532 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:15.453593 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d42f4353-efe7-44eb-93d9-86882ae4b532 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:15.481542 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:37:15.513390 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdb7ef3-e45c-44a7-adee-a7ddc25ac8dc None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:15.519401 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf45e13-0286-4bef-9e17-38777ae7d7e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:15.520238 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf45e13-0286-4bef-9e17-38777ae7d7e3 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:15.520599 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf45e13-0286-4bef-9e17-38777ae7d7e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:15.521246 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf45e13-0286-4bef-9e17-38777ae7d7e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:15.529427 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42f4353-efe7-44eb-93d9-86882ae4b532 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:15.531377 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42f4353-efe7-44eb-93d9-86882ae4b532 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:15.532393 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf45e13-0286-4bef-9e17-38777ae7d7e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:15.534572 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf45e13-0286-4bef-9e17-38777ae7d7e3 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:15.541378 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 929/1872] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:37:15 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 15:37:15.553428 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 944/1873] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:37:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 15:37:15.863146 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42f4353-efe7-44eb-93d9-86882ae4b532 service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:15.865374 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d42f4353-efe7-44eb-93d9-86882ae4b532 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:15.880382 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42f4353-efe7-44eb-93d9-86882ae4b532 service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:15.885791 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41cff55-079a-4350-9172-5ed92b5ee944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:15.886761 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41cff55-079a-4350-9172-5ed92b5ee944 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:15.887190 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41cff55-079a-4350-9172-5ed92b5ee944 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:15.887302 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41cff55-079a-4350-9172-5ed92b5ee944 admin admin] PATH_INFO: `/v3/projects/84580006f8ec446b8a5db16ce3e7db15` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:15.888589 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41cff55-079a-4350-9172-5ed92b5ee944 admin admin] RBAC: Authorizing `identity:delete_project(project_id=84580006f8ec446b8a5db16ce3e7db15)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:15.890923 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41cff55-079a-4350-9172-5ed92b5ee944 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:15.951187 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d41cff55-079a-4350-9172-5ed92b5ee944 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84580006f8ec446b8a5db16ce3e7db15', 'request_id': 'req-d41cff55-079a-4350-9172-5ed92b5ee944'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:15.957237 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d41cff55-079a-4350-9172-5ed92b5ee944 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84580006f8ec446b8a5db16ce3e7db15', 'request_id': 'req-d41cff55-079a-4350-9172-5ed92b5ee944'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:15.963486 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d41cff55-079a-4350-9172-5ed92b5ee944 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84580006f8ec446b8a5db16ce3e7db15', 'request_id': 'req-d41cff55-079a-4350-9172-5ed92b5ee944'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:15.978129 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d41cff55-079a-4350-9172-5ed92b5ee944 admin admin] The token cache is being invalidate because project 84580006f8ec446b8a5db16ce3e7db15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 15:37:15.978497 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d41cff55-079a-4350-9172-5ed92b5ee944 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:15.979382 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d41cff55-079a-4350-9172-5ed92b5ee944 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:15.981179 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 930/1874] 10.222.0.99 () {58 vars in 1232 bytes} [Thu Jul 14 15:37:15 2022] DELETE /identity/v3/projects/84580006f8ec446b8a5db16ce3e7db15 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:37:19.174357 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c0853a-0d9a-497d-80d4-6b7f7fd88a40 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:19.175409 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c0853a-0d9a-497d-80d4-6b7f7fd88a40 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:19.175888 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c0853a-0d9a-497d-80d4-6b7f7fd88a40 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:19.177408 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 945/1875] 10.222.0.99 () {54 vars in 921 bytes} [Thu Jul 14 15:37:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:37:19.186773 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be18699-c655-4df6-b7f3-3386917ceb4c None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:19.187694 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be18699-c655-4df6-b7f3-3386917ceb4c None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:19.187694 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be18699-c655-4df6-b7f3-3386917ceb4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:19.223982 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-3be18699-c655-4df6-b7f3-3386917ceb4c None None] MFA Rules not processed for user `3a9b1c2622934c5185f3b5f1ea623bc0`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:37:19.226533 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-3be18699-c655-4df6-b7f3-3386917ceb4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:19.353699 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 931/1876] 10.222.0.99 () {58 vars in 1018 bytes} [Thu Jul 14 15:37:19 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:37:19.367080 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c0853a-0d9a-497d-80d4-6b7f7fd88a40 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:19.369203 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-78c0853a-0d9a-497d-80d4-6b7f7fd88a40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:19.393034 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c0853a-0d9a-497d-80d4-6b7f7fd88a40 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:19.399317 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ca5ba8-824b-4145-95ba-2c2307f8b847 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9b1c2622934c5185f3b5f1ea623bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:19.400048 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ca5ba8-824b-4145-95ba-2c2307f8b847 service nova] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:19.400355 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ca5ba8-824b-4145-95ba-2c2307f8b847 service nova] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:19.400644 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ca5ba8-824b-4145-95ba-2c2307f8b847 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:19.410921 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ca5ba8-824b-4145-95ba-2c2307f8b847 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:19.412699 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ca5ba8-824b-4145-95ba-2c2307f8b847 service nova] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:19.435365 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 946/1877] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:37:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:37:22.259224 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be18699-c655-4df6-b7f3-3386917ceb4c None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:22.262639 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-3be18699-c655-4df6-b7f3-3386917ceb4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:22.325457 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:37:22.376950 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be18699-c655-4df6-b7f3-3386917ceb4c None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:22.383337 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae85f21-0843-44a8-b868-ca36ea1c6f1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:22.385223 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae85f21-0843-44a8-b868-ca36ea1c6f1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:22.386414 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae85f21-0843-44a8-b868-ca36ea1c6f1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:22.386872 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae85f21-0843-44a8-b868-ca36ea1c6f1c admin admin] PATH_INFO: `/v3/users/31c8fe96f0e64373b9c3d2b53a8da996` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:22.388747 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae85f21-0843-44a8-b868-ca36ea1c6f1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=31c8fe96f0e64373b9c3d2b53a8da996)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:22.392183 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae85f21-0843-44a8-b868-ca36ea1c6f1c admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:22.463768 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2ae85f21-0843-44a8-b868-ca36ea1c6f1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31c8fe96f0e64373b9c3d2b53a8da996'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:22.474133 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2ae85f21-0843-44a8-b868-ca36ea1c6f1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31c8fe96f0e64373b9c3d2b53a8da996'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:22.492749 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2ae85f21-0843-44a8-b868-ca36ea1c6f1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31c8fe96f0e64373b9c3d2b53a8da996'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:22.494177 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2ae85f21-0843-44a8-b868-ca36ea1c6f1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31c8fe96f0e64373b9c3d2b53a8da996'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:22.494870 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2ae85f21-0843-44a8-b868-ca36ea1c6f1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31c8fe96f0e64373b9c3d2b53a8da996'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:22.510041 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 932/1878] 10.222.0.99 () {58 vars in 1223 bytes} [Thu Jul 14 15:37:22 2022] DELETE /identity/v3/users/31c8fe96f0e64373b9c3d2b53a8da996 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:37:22.516454 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf228fef-e2a1-43cf-b3c1-8c63b66b2164 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:22.517809 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf228fef-e2a1-43cf-b3c1-8c63b66b2164 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:22.518446 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf228fef-e2a1-43cf-b3c1-8c63b66b2164 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:22.547919 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-bf228fef-e2a1-43cf-b3c1-8c63b66b2164 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:37:22.549423 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-bf228fef-e2a1-43cf-b3c1-8c63b66b2164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:22.635394 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 947/1879] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:37:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:37:22.650147 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae85f21-0843-44a8-b868-ca36ea1c6f1c admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:22.651680 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-2ae85f21-0843-44a8-b868-ca36ea1c6f1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:22.732053 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae85f21-0843-44a8-b868-ca36ea1c6f1c admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:22.738485 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d52e77a-72b1-46f5-9f7a-0405985677de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:22.739282 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d52e77a-72b1-46f5-9f7a-0405985677de service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:22.741370 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d52e77a-72b1-46f5-9f7a-0405985677de service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:22.741679 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d52e77a-72b1-46f5-9f7a-0405985677de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:22.761014 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d52e77a-72b1-46f5-9f7a-0405985677de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:22.762420 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d52e77a-72b1-46f5-9f7a-0405985677de service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:22.781201 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 933/1880] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:37:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:37:23.526427 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf228fef-e2a1-43cf-b3c1-8c63b66b2164 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:23.528083 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-bf228fef-e2a1-43cf-b3c1-8c63b66b2164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:23.542131 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf228fef-e2a1-43cf-b3c1-8c63b66b2164 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:23.547728 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0ba43e-a1f1-4ecc-a566-ba00102efd0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:23.548686 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0ba43e-a1f1-4ecc-a566-ba00102efd0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:23.549046 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0ba43e-a1f1-4ecc-a566-ba00102efd0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:23.549402 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0ba43e-a1f1-4ecc-a566-ba00102efd0e admin admin] PATH_INFO: `/v3/projects/03112c7f482f4bc38dde9bae2e5fe9e3` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:23.550761 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0ba43e-a1f1-4ecc-a566-ba00102efd0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=03112c7f482f4bc38dde9bae2e5fe9e3)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:23.553222 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0ba43e-a1f1-4ecc-a566-ba00102efd0e admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:23.602345 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-4a0ba43e-a1f1-4ecc-a566-ba00102efd0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03112c7f482f4bc38dde9bae2e5fe9e3', 'request_id': 'req-4a0ba43e-a1f1-4ecc-a566-ba00102efd0e'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:23.610073 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-4a0ba43e-a1f1-4ecc-a566-ba00102efd0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03112c7f482f4bc38dde9bae2e5fe9e3', 'request_id': 'req-4a0ba43e-a1f1-4ecc-a566-ba00102efd0e'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:23.616688 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-4a0ba43e-a1f1-4ecc-a566-ba00102efd0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03112c7f482f4bc38dde9bae2e5fe9e3', 'request_id': 'req-4a0ba43e-a1f1-4ecc-a566-ba00102efd0e'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:23.631082 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-4a0ba43e-a1f1-4ecc-a566-ba00102efd0e admin admin] The token cache is being invalidate because project 03112c7f482f4bc38dde9bae2e5fe9e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 15:37:23.631492 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-4a0ba43e-a1f1-4ecc-a566-ba00102efd0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:23.632241 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-4a0ba43e-a1f1-4ecc-a566-ba00102efd0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:23.634366 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 948/1881] 10.222.0.99 () {58 vars in 1232 bytes} [Thu Jul 14 15:37:23 2022] DELETE /identity/v3/projects/03112c7f482f4bc38dde9bae2e5fe9e3 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:37:23.638146 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d52e77a-72b1-46f5-9f7a-0405985677de service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:23.651408 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d52e77a-72b1-46f5-9f7a-0405985677de service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:23.713980 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:37:23.722916 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe60afd3-5669-4979-a4a5-9dbd4651a142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:23.723848 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe60afd3-5669-4979-a4a5-9dbd4651a142 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:23.724333 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe60afd3-5669-4979-a4a5-9dbd4651a142 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:23.724783 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe60afd3-5669-4979-a4a5-9dbd4651a142 admin admin] PATH_INFO: `/v3/users/705e38154b7740a2a85a69f138c98388` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:23.726617 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe60afd3-5669-4979-a4a5-9dbd4651a142 admin admin] RBAC: Authorizing `identity:delete_user(user_id=705e38154b7740a2a85a69f138c98388)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:23.729504 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe60afd3-5669-4979-a4a5-9dbd4651a142 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:23.786879 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-fe60afd3-5669-4979-a4a5-9dbd4651a142 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '705e38154b7740a2a85a69f138c98388'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:23.805876 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-fe60afd3-5669-4979-a4a5-9dbd4651a142 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '705e38154b7740a2a85a69f138c98388'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:23.819775 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-fe60afd3-5669-4979-a4a5-9dbd4651a142 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '705e38154b7740a2a85a69f138c98388'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:23.820446 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-fe60afd3-5669-4979-a4a5-9dbd4651a142 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '705e38154b7740a2a85a69f138c98388'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:23.821062 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-fe60afd3-5669-4979-a4a5-9dbd4651a142 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '705e38154b7740a2a85a69f138c98388'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:23.833968 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 934/1882] 10.222.0.99 () {58 vars in 1223 bytes} [Thu Jul 14 15:37:23 2022] DELETE /identity/v3/users/705e38154b7740a2a85a69f138c98388 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 15:37:24.107890 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0ba43e-a1f1-4ecc-a566-ba00102efd0e admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:24.109542 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-4a0ba43e-a1f1-4ecc-a566-ba00102efd0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:24.117763 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:37:24.123379 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0ba43e-a1f1-4ecc-a566-ba00102efd0e admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:24.184514 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c22102-5c59-4066-bc3f-c0b5793c4e2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:24.185386 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c22102-5c59-4066-bc3f-c0b5793c4e2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:24.185838 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c22102-5c59-4066-bc3f-c0b5793c4e2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:24.185992 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c22102-5c59-4066-bc3f-c0b5793c4e2a admin admin] PATH_INFO: `/v3/projects/ebaf3a1313a74551bf44a8134ea1f792` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:24.187749 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c22102-5c59-4066-bc3f-c0b5793c4e2a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ebaf3a1313a74551bf44a8134ea1f792)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:24.189692 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c22102-5c59-4066-bc3f-c0b5793c4e2a admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:24.239023 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-87c22102-5c59-4066-bc3f-c0b5793c4e2a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ebaf3a1313a74551bf44a8134ea1f792', 'request_id': 'req-87c22102-5c59-4066-bc3f-c0b5793c4e2a'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:24.245470 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-87c22102-5c59-4066-bc3f-c0b5793c4e2a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ebaf3a1313a74551bf44a8134ea1f792', 'request_id': 'req-87c22102-5c59-4066-bc3f-c0b5793c4e2a'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:24.252428 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-87c22102-5c59-4066-bc3f-c0b5793c4e2a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ebaf3a1313a74551bf44a8134ea1f792', 'request_id': 'req-87c22102-5c59-4066-bc3f-c0b5793c4e2a'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:24.267775 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-87c22102-5c59-4066-bc3f-c0b5793c4e2a admin admin] The token cache is being invalidate because project ebaf3a1313a74551bf44a8134ea1f792 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 15:37:24.268072 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-87c22102-5c59-4066-bc3f-c0b5793c4e2a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:24.268743 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-87c22102-5c59-4066-bc3f-c0b5793c4e2a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:24.270575 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 949/1883] 10.222.0.99 () {58 vars in 1232 bytes} [Thu Jul 14 15:37:24 2022] DELETE /identity/v3/projects/ebaf3a1313a74551bf44a8134ea1f792 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:37:24.275667 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceb4b83-97af-4a92-a6e6-4fdbf23edeea None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:24.275977 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceb4b83-97af-4a92-a6e6-4fdbf23edeea None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:24.276266 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceb4b83-97af-4a92-a6e6-4fdbf23edeea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:24.295326 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-0ceb4b83-97af-4a92-a6e6-4fdbf23edeea None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:37:24.296773 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0ceb4b83-97af-4a92-a6e6-4fdbf23edeea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:24.371053 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 935/1884] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:37:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:37:24.378889 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483c9395-98b0-4be5-956b-11474197e57d None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:24.379182 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483c9395-98b0-4be5-956b-11474197e57d None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:24.379486 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483c9395-98b0-4be5-956b-11474197e57d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:24.399889 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-483c9395-98b0-4be5-956b-11474197e57d None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:37:24.401103 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-483c9395-98b0-4be5-956b-11474197e57d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:24.410315 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 950/1885] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:37:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:37:24.415018 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceb4b83-97af-4a92-a6e6-4fdbf23edeea None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:24.425400 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceb4b83-97af-4a92-a6e6-4fdbf23edeea None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:24.432540 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35395b0-e228-42b4-83cd-05cdadcb8ab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:24.433221 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35395b0-e228-42b4-83cd-05cdadcb8ab2 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:24.433515 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35395b0-e228-42b4-83cd-05cdadcb8ab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:24.433802 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35395b0-e228-42b4-83cd-05cdadcb8ab2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:24.434540 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35395b0-e228-42b4-83cd-05cdadcb8ab2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:37:24.434855 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35395b0-e228-42b4-83cd-05cdadcb8ab2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:24.436593 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35395b0-e228-42b4-83cd-05cdadcb8ab2 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:24.450120 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 936/1886] 10.222.0.99 () {58 vars in 1152 bytes} [Thu Jul 14 15:37:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:37:24.455088 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483c9395-98b0-4be5-956b-11474197e57d None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:24.464542 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483c9395-98b0-4be5-956b-11474197e57d None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:24.469970 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af4ad01-0ebc-4da3-a0d5-a7382bd40975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:24.470627 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af4ad01-0ebc-4da3-a0d5-a7382bd40975 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:24.470917 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af4ad01-0ebc-4da3-a0d5-a7382bd40975 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:24.471197 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af4ad01-0ebc-4da3-a0d5-a7382bd40975 admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:24.471931 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af4ad01-0ebc-4da3-a0d5-a7382bd40975 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:24.473849 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af4ad01-0ebc-4da3-a0d5-a7382bd40975 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:24.501372 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 951/1887] 10.222.0.99 () {60 vars in 1152 bytes} [Thu Jul 14 15:37:24 2022] POST /identity/v3/projects => generated 367 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:37:24.505621 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35395b0-e228-42b4-83cd-05cdadcb8ab2 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:24.515514 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35395b0-e228-42b4-83cd-05cdadcb8ab2 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:24.585188 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f77687c-4e3a-40a1-a77f-a73d20dd0221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:24.586019 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f77687c-4e3a-40a1-a77f-a73d20dd0221 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:24.586387 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f77687c-4e3a-40a1-a77f-a73d20dd0221 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:24.586731 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f77687c-4e3a-40a1-a77f-a73d20dd0221 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:24.587699 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f77687c-4e3a-40a1-a77f-a73d20dd0221 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:24.589708 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f77687c-4e3a-40a1-a77f-a73d20dd0221 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:24.598687 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:37:24.621565 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 937/1888] 10.222.0.99 () {60 vars in 1143 bytes} [Thu Jul 14 15:37:24 2022] POST /identity/v3/users => generated 284 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:37:24.623368 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af4ad01-0ebc-4da3-a0d5-a7382bd40975 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:24.641191 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af4ad01-0ebc-4da3-a0d5-a7382bd40975 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:24.647813 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f06814-c134-43b8-957d-cf89cf699d61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:24.648507 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f06814-c134-43b8-957d-cf89cf699d61 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:24.648793 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f06814-c134-43b8-957d-cf89cf699d61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:24.649077 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f06814-c134-43b8-957d-cf89cf699d61 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:24.650313 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f06814-c134-43b8-957d-cf89cf699d61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:24.652466 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f06814-c134-43b8-957d-cf89cf699d61 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:24.668388 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 952/1889] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:37:24 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:37:24.673585 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f77687c-4e3a-40a1-a77f-a73d20dd0221 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:24.684815 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f77687c-4e3a-40a1-a77f-a73d20dd0221 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:24.691364 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b6127c-b8de-4adc-9075-b91f200f1021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:24.692077 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b6127c-b8de-4adc-9075-b91f200f1021 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:24.692363 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b6127c-b8de-4adc-9075-b91f200f1021 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:24.692641 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b6127c-b8de-4adc-9075-b91f200f1021 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:24.693377 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b6127c-b8de-4adc-9075-b91f200f1021 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:24.694988 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b6127c-b8de-4adc-9075-b91f200f1021 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:24.708438 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 938/1890] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:37:24 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:37:24.713283 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f06814-c134-43b8-957d-cf89cf699d61 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:24.723852 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f06814-c134-43b8-957d-cf89cf699d61 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:24.729448 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaea6d9-7a9e-439e-baec-fc203c589684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:24.730352 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaea6d9-7a9e-439e-baec-fc203c589684 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:24.730641 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaea6d9-7a9e-439e-baec-fc203c589684 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:24.730920 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaea6d9-7a9e-439e-baec-fc203c589684 admin admin] PATH_INFO: `/v3/projects/e9f34fbe8236468da32a47c3a1c5d5e3/users/ca2b71946b45425f9396c10ebe7866df/roles/b8c98fddc0fd4436b577fd1d3361bc38` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:24.749634 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caaea6d9-7a9e-439e-baec-fc203c589684 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9f34fbe8236468da32a47c3a1c5d5e3, user_id=ca2b71946b45425f9396c10ebe7866df, role_id=b8c98fddc0fd4436b577fd1d3361bc38)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:24.752701 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caaea6d9-7a9e-439e-baec-fc203c589684 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:24.768784 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 953/1891] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 15:37:24 2022] PUT /identity/v3/projects/e9f34fbe8236468da32a47c3a1c5d5e3/users/ca2b71946b45425f9396c10ebe7866df/roles/b8c98fddc0fd4436b577fd1d3361bc38 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:37:24.785137 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ccf636-41d5-4463-b84e-3cfeccfc6bd9 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:24.785522 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ccf636-41d5-4463-b84e-3cfeccfc6bd9 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:24.785920 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ccf636-41d5-4463-b84e-3cfeccfc6bd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:24.821784 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-e2ccf636-41d5-4463-b84e-3cfeccfc6bd9 None None] MFA Rules not processed for user `ca2b71946b45425f9396c10ebe7866df`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:37:24.823208 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e2ccf636-41d5-4463-b84e-3cfeccfc6bd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:24.918442 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 939/1892] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:37:24 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:37:24.923286 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaea6d9-7a9e-439e-baec-fc203c589684 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:24.967401 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:37:25.022515 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaea6d9-7a9e-439e-baec-fc203c589684 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:25.027350 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577e0033-26e0-4f31-820c-c32a3d807c73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:25.028052 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577e0033-26e0-4f31-820c-c32a3d807c73 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:25.028344 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577e0033-26e0-4f31-820c-c32a3d807c73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:25.028626 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577e0033-26e0-4f31-820c-c32a3d807c73 admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:25.029321 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577e0033-26e0-4f31-820c-c32a3d807c73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:25.031815 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577e0033-26e0-4f31-820c-c32a3d807c73 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:25.057907 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 954/1893] 10.222.0.99 () {60 vars in 1152 bytes} [Thu Jul 14 15:37:24 2022] POST /identity/v3/projects => generated 363 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:37:25.062386 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ccf636-41d5-4463-b84e-3cfeccfc6bd9 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:25.074055 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ccf636-41d5-4463-b84e-3cfeccfc6bd9 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:25.142615 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ac7d43-d3c5-4441-a23f-5512e459fb65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:25.143307 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ac7d43-d3c5-4441-a23f-5512e459fb65 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:25.143637 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ac7d43-d3c5-4441-a23f-5512e459fb65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:25.144022 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ac7d43-d3c5-4441-a23f-5512e459fb65 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:25.144777 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ac7d43-d3c5-4441-a23f-5512e459fb65 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:25.147043 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ac7d43-d3c5-4441-a23f-5512e459fb65 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:25.177097 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 940/1894] 10.222.0.99 () {60 vars in 1143 bytes} [Thu Jul 14 15:37:25 2022] POST /identity/v3/users => generated 288 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 15:37:25.181348 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577e0033-26e0-4f31-820c-c32a3d807c73 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:25.196088 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577e0033-26e0-4f31-820c-c32a3d807c73 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:25.202673 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e53d613-5b30-4754-8aad-b1937bc081b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:25.203405 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e53d613-5b30-4754-8aad-b1937bc081b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:25.203727 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e53d613-5b30-4754-8aad-b1937bc081b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:25.204009 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e53d613-5b30-4754-8aad-b1937bc081b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:25.204756 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e53d613-5b30-4754-8aad-b1937bc081b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:25.206424 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e53d613-5b30-4754-8aad-b1937bc081b7 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:25.219406 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 955/1895] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:37:25 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:37:25.223679 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ac7d43-d3c5-4441-a23f-5512e459fb65 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:25.235024 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ac7d43-d3c5-4441-a23f-5512e459fb65 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:25.240313 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a860c4-d5a9-4a22-9da6-edd405ab55ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:25.241471 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a860c4-d5a9-4a22-9da6-edd405ab55ad admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:25.241984 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a860c4-d5a9-4a22-9da6-edd405ab55ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:25.242473 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a860c4-d5a9-4a22-9da6-edd405ab55ad admin admin] PATH_INFO: `/v3/projects/3ab11ee7c2264fe587ccf3388de2b854/users/53fb4a5d1c4f4d4ca3d0c220e492755a/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:25.261187 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a860c4-d5a9-4a22-9da6-edd405ab55ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ab11ee7c2264fe587ccf3388de2b854, user_id=53fb4a5d1c4f4d4ca3d0c220e492755a, role_id=d0f6ba1314f14d9f9c816bdb5d0c3aac)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:25.263938 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a860c4-d5a9-4a22-9da6-edd405ab55ad admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:25.287459 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 941/1896] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 15:37:25 2022] PUT /identity/v3/projects/3ab11ee7c2264fe587ccf3388de2b854/users/53fb4a5d1c4f4d4ca3d0c220e492755a/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:37:25.291762 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e53d613-5b30-4754-8aad-b1937bc081b7 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:25.304686 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e53d613-5b30-4754-8aad-b1937bc081b7 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:25.373715 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6816104b-9eba-4646-8515-24be091525bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:25.374528 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6816104b-9eba-4646-8515-24be091525bf admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:25.376175 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6816104b-9eba-4646-8515-24be091525bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:25.376175 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6816104b-9eba-4646-8515-24be091525bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:25.376175 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6816104b-9eba-4646-8515-24be091525bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:25.377645 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6816104b-9eba-4646-8515-24be091525bf admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:25.388094 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 956/1897] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:37:25 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:37:25.393584 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a860c4-d5a9-4a22-9da6-edd405ab55ad admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:25.404978 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a860c4-d5a9-4a22-9da6-edd405ab55ad admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:25.411116 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b56971-d09f-4985-9832-a84922bc62f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:25.412070 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b56971-d09f-4985-9832-a84922bc62f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:25.412364 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b56971-d09f-4985-9832-a84922bc62f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:25.412647 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b56971-d09f-4985-9832-a84922bc62f3 admin admin] PATH_INFO: `/v3/projects/3ab11ee7c2264fe587ccf3388de2b854/users/53fb4a5d1c4f4d4ca3d0c220e492755a/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:25.414628 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b56971-d09f-4985-9832-a84922bc62f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ab11ee7c2264fe587ccf3388de2b854, user_id=53fb4a5d1c4f4d4ca3d0c220e492755a, role_id=d0f6ba1314f14d9f9c816bdb5d0c3aac)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:25.417622 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b56971-d09f-4985-9832-a84922bc62f3 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:25.435667 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 942/1898] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 15:37:25 2022] PUT /identity/v3/projects/3ab11ee7c2264fe587ccf3388de2b854/users/53fb4a5d1c4f4d4ca3d0c220e492755a/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:37:25.441874 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bf3abe-2eca-40fb-bda8-fd2bbb94b493 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:25.442084 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bf3abe-2eca-40fb-bda8-fd2bbb94b493 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:25.442365 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bf3abe-2eca-40fb-bda8-fd2bbb94b493 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:25.488262 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-07bf3abe-2eca-40fb-bda8-fd2bbb94b493 None None] MFA Rules not processed for user `53fb4a5d1c4f4d4ca3d0c220e492755a`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:37:25.489945 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-07bf3abe-2eca-40fb-bda8-fd2bbb94b493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:25.587255 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 957/1899] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:37:25 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:37:25.599529 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b56971-d09f-4985-9832-a84922bc62f3 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:25.601539 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-30b56971-d09f-4985-9832-a84922bc62f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:25.630037 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:37:25.674512 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b56971-d09f-4985-9832-a84922bc62f3 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:25.678284 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52943c38-c4d8-460e-ac44-627e9b600131 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3e1a2b83f3d4d88ba5084dfaffd22f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:25.679083 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52943c38-c4d8-460e-ac44-627e9b600131 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:25.679367 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52943c38-c4d8-460e-ac44-627e9b600131 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:25.679704 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52943c38-c4d8-460e-ac44-627e9b600131 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:25.690603 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52943c38-c4d8-460e-ac44-627e9b600131 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:25.692225 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52943c38-c4d8-460e-ac44-627e9b600131 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:25.702669 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 943/1900] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:37:25 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:37:27.363682 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bf3abe-2eca-40fb-bda8-fd2bbb94b493 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:27.394532 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:37:27.434985 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bf3abe-2eca-40fb-bda8-fd2bbb94b493 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:27.440618 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e415a3ed-fcbe-4b46-9854-d73a8ea160df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:27.441490 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e415a3ed-fcbe-4b46-9854-d73a8ea160df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:27.441793 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e415a3ed-fcbe-4b46-9854-d73a8ea160df admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:27.442069 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e415a3ed-fcbe-4b46-9854-d73a8ea160df admin admin] PATH_INFO: `/v3/users/ca2b71946b45425f9396c10ebe7866df` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:27.443280 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e415a3ed-fcbe-4b46-9854-d73a8ea160df admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca2b71946b45425f9396c10ebe7866df)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:27.445308 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e415a3ed-fcbe-4b46-9854-d73a8ea160df admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:27.496665 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-e415a3ed-fcbe-4b46-9854-d73a8ea160df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca2b71946b45425f9396c10ebe7866df'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:27.505093 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-e415a3ed-fcbe-4b46-9854-d73a8ea160df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca2b71946b45425f9396c10ebe7866df'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:27.518403 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-e415a3ed-fcbe-4b46-9854-d73a8ea160df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca2b71946b45425f9396c10ebe7866df'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:27.519098 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-e415a3ed-fcbe-4b46-9854-d73a8ea160df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca2b71946b45425f9396c10ebe7866df'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:27.519740 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-e415a3ed-fcbe-4b46-9854-d73a8ea160df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca2b71946b45425f9396c10ebe7866df'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:27.533864 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 958/1901] 10.222.0.99 () {58 vars in 1223 bytes} [Thu Jul 14 15:37:27 2022] DELETE /identity/v3/users/ca2b71946b45425f9396c10ebe7866df => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:37:27.538280 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24298019-4b1b-400d-9ab6-698cd1c1789c None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:27.538841 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24298019-4b1b-400d-9ab6-698cd1c1789c None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:27.538937 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24298019-4b1b-400d-9ab6-698cd1c1789c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:27.559038 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-24298019-4b1b-400d-9ab6-698cd1c1789c None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:37:27.560511 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-24298019-4b1b-400d-9ab6-698cd1c1789c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:27.652405 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 944/1902] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:37:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 15:37:27.665464 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e415a3ed-fcbe-4b46-9854-d73a8ea160df admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:27.666975 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-e415a3ed-fcbe-4b46-9854-d73a8ea160df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:27.752190 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e415a3ed-fcbe-4b46-9854-d73a8ea160df admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:27.756273 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507906af-8862-4c0a-8000-48a6d21113eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:27.757096 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507906af-8862-4c0a-8000-48a6d21113eb service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:27.757384 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507906af-8862-4c0a-8000-48a6d21113eb service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:27.757710 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507906af-8862-4c0a-8000-48a6d21113eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:27.771053 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507906af-8862-4c0a-8000-48a6d21113eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:27.773264 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507906af-8862-4c0a-8000-48a6d21113eb service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:27.787208 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 959/1903] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:37:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:37:27.939829 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24298019-4b1b-400d-9ab6-698cd1c1789c None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:27.941320 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-24298019-4b1b-400d-9ab6-698cd1c1789c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:27.963020 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24298019-4b1b-400d-9ab6-698cd1c1789c None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:27.976126 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6d8b20-a40e-4281-9c55-ee3ca41ef1d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:27.977044 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6d8b20-a40e-4281-9c55-ee3ca41ef1d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:27.977335 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6d8b20-a40e-4281-9c55-ee3ca41ef1d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:27.977618 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6d8b20-a40e-4281-9c55-ee3ca41ef1d4 admin admin] PATH_INFO: `/v3/users/8f8b17036d4d4f04976f9a203258347b` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:27.982453 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6d8b20-a40e-4281-9c55-ee3ca41ef1d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f8b17036d4d4f04976f9a203258347b)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:27.986912 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6d8b20-a40e-4281-9c55-ee3ca41ef1d4 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:28.125638 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-6e6d8b20-a40e-4281-9c55-ee3ca41ef1d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f8b17036d4d4f04976f9a203258347b'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:28.142377 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-6e6d8b20-a40e-4281-9c55-ee3ca41ef1d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f8b17036d4d4f04976f9a203258347b'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:28.163153 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-6e6d8b20-a40e-4281-9c55-ee3ca41ef1d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f8b17036d4d4f04976f9a203258347b'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:28.164060 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-6e6d8b20-a40e-4281-9c55-ee3ca41ef1d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f8b17036d4d4f04976f9a203258347b'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:28.164673 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-6e6d8b20-a40e-4281-9c55-ee3ca41ef1d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f8b17036d4d4f04976f9a203258347b'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:28.186355 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 945/1904] 10.222.0.99 () {58 vars in 1223 bytes} [Thu Jul 14 15:37:27 2022] DELETE /identity/v3/users/8f8b17036d4d4f04976f9a203258347b => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:37:28.188753 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce87f4c-9a6a-4c7a-8318-42c8768b6e0a None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:28.189118 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce87f4c-9a6a-4c7a-8318-42c8768b6e0a None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:28.189496 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce87f4c-9a6a-4c7a-8318-42c8768b6e0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:28.248973 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-1ce87f4c-9a6a-4c7a-8318-42c8768b6e0a None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:37:28.274655 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-1ce87f4c-9a6a-4c7a-8318-42c8768b6e0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:28.396452 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:37:28.423263 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 960/1905] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:37:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:37:28.434767 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6d8b20-a40e-4281-9c55-ee3ca41ef1d4 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:28.436883 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-6e6d8b20-a40e-4281-9c55-ee3ca41ef1d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:28.453306 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce87f4c-9a6a-4c7a-8318-42c8768b6e0a None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:28.455665 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-1ce87f4c-9a6a-4c7a-8318-42c8768b6e0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:28.495905 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce87f4c-9a6a-4c7a-8318-42c8768b6e0a None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:28.504300 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:37:28.507088 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddffd532-3f20-4f53-9168-81284d023cbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:28.507961 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddffd532-3f20-4f53-9168-81284d023cbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:28.508397 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddffd532-3f20-4f53-9168-81284d023cbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:28.508757 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddffd532-3f20-4f53-9168-81284d023cbf admin admin] PATH_INFO: `/v3/projects/e9f34fbe8236468da32a47c3a1c5d5e3` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:28.511419 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddffd532-3f20-4f53-9168-81284d023cbf admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9f34fbe8236468da32a47c3a1c5d5e3)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:28.516265 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddffd532-3f20-4f53-9168-81284d023cbf admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:28.539131 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6d8b20-a40e-4281-9c55-ee3ca41ef1d4 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:28.544249 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23af3e1f-d7a6-4645-8a47-98da31f4413f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:28.545131 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23af3e1f-d7a6-4645-8a47-98da31f4413f service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:28.546064 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23af3e1f-d7a6-4645-8a47-98da31f4413f service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:28.546508 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23af3e1f-d7a6-4645-8a47-98da31f4413f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:28.560093 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23af3e1f-d7a6-4645-8a47-98da31f4413f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:28.562324 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23af3e1f-d7a6-4645-8a47-98da31f4413f service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:28.574921 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 946/1906] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:37:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 15:37:28.600985 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-ddffd532-3f20-4f53-9168-81284d023cbf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9f34fbe8236468da32a47c3a1c5d5e3', 'request_id': 'req-ddffd532-3f20-4f53-9168-81284d023cbf'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:28.612478 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-ddffd532-3f20-4f53-9168-81284d023cbf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9f34fbe8236468da32a47c3a1c5d5e3', 'request_id': 'req-ddffd532-3f20-4f53-9168-81284d023cbf'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:28.627427 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-ddffd532-3f20-4f53-9168-81284d023cbf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9f34fbe8236468da32a47c3a1c5d5e3', 'request_id': 'req-ddffd532-3f20-4f53-9168-81284d023cbf'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:28.657376 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-ddffd532-3f20-4f53-9168-81284d023cbf admin admin] The token cache is being invalidate because project e9f34fbe8236468da32a47c3a1c5d5e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 15:37:28.657702 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-ddffd532-3f20-4f53-9168-81284d023cbf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:28.663200 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-ddffd532-3f20-4f53-9168-81284d023cbf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:28.664975 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 961/1907] 10.222.0.99 () {58 vars in 1232 bytes} [Thu Jul 14 15:37:28 2022] DELETE /identity/v3/projects/e9f34fbe8236468da32a47c3a1c5d5e3 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:37:28.669168 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23af3e1f-d7a6-4645-8a47-98da31f4413f service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:28.684611 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23af3e1f-d7a6-4645-8a47-98da31f4413f service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:28.790389 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01053fda-9f32-4f8c-a713-16c9bd3c37eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:28.791370 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01053fda-9f32-4f8c-a713-16c9bd3c37eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:28.791817 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01053fda-9f32-4f8c-a713-16c9bd3c37eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:28.792107 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01053fda-9f32-4f8c-a713-16c9bd3c37eb admin admin] PATH_INFO: `/v3/users/53fb4a5d1c4f4d4ca3d0c220e492755a` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:28.793549 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01053fda-9f32-4f8c-a713-16c9bd3c37eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=53fb4a5d1c4f4d4ca3d0c220e492755a)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:28.796039 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01053fda-9f32-4f8c-a713-16c9bd3c37eb admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:28.858349 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-01053fda-9f32-4f8c-a713-16c9bd3c37eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53fb4a5d1c4f4d4ca3d0c220e492755a'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:28.871778 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-01053fda-9f32-4f8c-a713-16c9bd3c37eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53fb4a5d1c4f4d4ca3d0c220e492755a'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:28.897397 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-01053fda-9f32-4f8c-a713-16c9bd3c37eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53fb4a5d1c4f4d4ca3d0c220e492755a'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:28.898178 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-01053fda-9f32-4f8c-a713-16c9bd3c37eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53fb4a5d1c4f4d4ca3d0c220e492755a'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:28.899133 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-01053fda-9f32-4f8c-a713-16c9bd3c37eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53fb4a5d1c4f4d4ca3d0c220e492755a'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:28.920230 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 947/1908] 10.222.0.99 () {58 vars in 1223 bytes} [Thu Jul 14 15:37:28 2022] DELETE /identity/v3/users/53fb4a5d1c4f4d4ca3d0c220e492755a => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:37:29.168722 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddffd532-3f20-4f53-9168-81284d023cbf admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:29.170171 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-ddffd532-3f20-4f53-9168-81284d023cbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:29.184000 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddffd532-3f20-4f53-9168-81284d023cbf admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:29.264034 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b39341-a48b-4bd2-9fb7-4fe563927f82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:29.265314 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b39341-a48b-4bd2-9fb7-4fe563927f82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:29.265552 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b39341-a48b-4bd2-9fb7-4fe563927f82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:29.265796 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b39341-a48b-4bd2-9fb7-4fe563927f82 admin admin] PATH_INFO: `/v3/projects/8b724f3c688144a09418b78827eebe65` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:29.267073 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b39341-a48b-4bd2-9fb7-4fe563927f82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b724f3c688144a09418b78827eebe65)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:29.269208 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b39341-a48b-4bd2-9fb7-4fe563927f82 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:29.278603 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01053fda-9f32-4f8c-a713-16c9bd3c37eb admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:29.281252 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-01053fda-9f32-4f8c-a713-16c9bd3c37eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:29.303445 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01053fda-9f32-4f8c-a713-16c9bd3c37eb admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:29.310448 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf53a1b5-0eed-46f0-b8c4-1efa76358d2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:29.311244 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf53a1b5-0eed-46f0-b8c4-1efa76358d2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:29.312362 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf53a1b5-0eed-46f0-b8c4-1efa76358d2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:29.312694 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf53a1b5-0eed-46f0-b8c4-1efa76358d2a admin admin] PATH_INFO: `/v3/projects/3ab11ee7c2264fe587ccf3388de2b854` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:29.313945 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf53a1b5-0eed-46f0-b8c4-1efa76358d2a admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ab11ee7c2264fe587ccf3388de2b854)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:29.315937 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf53a1b5-0eed-46f0-b8c4-1efa76358d2a admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:29.323574 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-79b39341-a48b-4bd2-9fb7-4fe563927f82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b724f3c688144a09418b78827eebe65', 'request_id': 'req-79b39341-a48b-4bd2-9fb7-4fe563927f82'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:29.330867 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-79b39341-a48b-4bd2-9fb7-4fe563927f82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b724f3c688144a09418b78827eebe65', 'request_id': 'req-79b39341-a48b-4bd2-9fb7-4fe563927f82'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:29.336990 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-79b39341-a48b-4bd2-9fb7-4fe563927f82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b724f3c688144a09418b78827eebe65', 'request_id': 'req-79b39341-a48b-4bd2-9fb7-4fe563927f82'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:29.361267 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:37:29.368641 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-79b39341-a48b-4bd2-9fb7-4fe563927f82 admin admin] The token cache is being invalidate because project 8b724f3c688144a09418b78827eebe65 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 15:37:29.369290 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-79b39341-a48b-4bd2-9fb7-4fe563927f82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:29.371280 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-79b39341-a48b-4bd2-9fb7-4fe563927f82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:29.371898 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-cf53a1b5-0eed-46f0-b8c4-1efa76358d2a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ab11ee7c2264fe587ccf3388de2b854', 'request_id': 'req-cf53a1b5-0eed-46f0-b8c4-1efa76358d2a'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:29.373584 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 962/1909] 10.222.0.99 () {58 vars in 1232 bytes} [Thu Jul 14 15:37:29 2022] DELETE /identity/v3/projects/8b724f3c688144a09418b78827eebe65 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:37:29.379004 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b39341-a48b-4bd2-9fb7-4fe563927f82 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:29.381482 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-79b39341-a48b-4bd2-9fb7-4fe563927f82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:29.384076 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-cf53a1b5-0eed-46f0-b8c4-1efa76358d2a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ab11ee7c2264fe587ccf3388de2b854', 'request_id': 'req-cf53a1b5-0eed-46f0-b8c4-1efa76358d2a'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:29.386381 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:37:29.391417 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-cf53a1b5-0eed-46f0-b8c4-1efa76358d2a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ab11ee7c2264fe587ccf3388de2b854', 'request_id': 'req-cf53a1b5-0eed-46f0-b8c4-1efa76358d2a'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:29.415261 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-cf53a1b5-0eed-46f0-b8c4-1efa76358d2a admin admin] The token cache is being invalidate because project 3ab11ee7c2264fe587ccf3388de2b854 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 15:37:29.415816 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-cf53a1b5-0eed-46f0-b8c4-1efa76358d2a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:29.416695 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-cf53a1b5-0eed-46f0-b8c4-1efa76358d2a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:29.419778 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 948/1910] 10.222.0.99 () {58 vars in 1232 bytes} [Thu Jul 14 15:37:29 2022] DELETE /identity/v3/projects/3ab11ee7c2264fe587ccf3388de2b854 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:37:29.425769 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe31be4-201a-4571-a6b3-681414d98897 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:29.426070 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe31be4-201a-4571-a6b3-681414d98897 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:29.426357 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe31be4-201a-4571-a6b3-681414d98897 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:29.447122 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-bbe31be4-201a-4571-a6b3-681414d98897 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:37:29.448953 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-bbe31be4-201a-4571-a6b3-681414d98897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:29.493484 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b39341-a48b-4bd2-9fb7-4fe563927f82 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:29.614365 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 949/1911] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:37:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 15:37:29.628060 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b799854e-5025-48f3-b81e-5c1a50be47e8 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:29.628270 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b799854e-5025-48f3-b81e-5c1a50be47e8 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:29.628562 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b799854e-5025-48f3-b81e-5c1a50be47e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:29.642724 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce867b0-771b-4fd3-a980-2c5c02950f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:29.643870 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce867b0-771b-4fd3-a980-2c5c02950f2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:29.644267 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce867b0-771b-4fd3-a980-2c5c02950f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:29.644678 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce867b0-771b-4fd3-a980-2c5c02950f2b admin admin] PATH_INFO: `/v3/users/8cb60118608742c1bfe3c0f93a9d99b9` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:29.646700 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce867b0-771b-4fd3-a980-2c5c02950f2b admin admin] RBAC: Authorizing `identity:delete_user(user_id=8cb60118608742c1bfe3c0f93a9d99b9)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:29.649927 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce867b0-771b-4fd3-a980-2c5c02950f2b admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:29.709872 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-b799854e-5025-48f3-b81e-5c1a50be47e8 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:37:29.714597 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b799854e-5025-48f3-b81e-5c1a50be47e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:29.736389 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 950/1912] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:37:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:37:29.741238 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-1ce867b0-771b-4fd3-a980-2c5c02950f2b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8cb60118608742c1bfe3c0f93a9d99b9'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:29.742426 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b799854e-5025-48f3-b81e-5c1a50be47e8 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:29.767227 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-1ce867b0-771b-4fd3-a980-2c5c02950f2b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8cb60118608742c1bfe3c0f93a9d99b9'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:29.773262 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b799854e-5025-48f3-b81e-5c1a50be47e8 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:29.788724 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7452246f-4b3a-4e09-ad26-9b04f450bb7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:29.790500 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-1ce867b0-771b-4fd3-a980-2c5c02950f2b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8cb60118608742c1bfe3c0f93a9d99b9'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:29.791434 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7452246f-4b3a-4e09-ad26-9b04f450bb7f admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:29.792440 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-1ce867b0-771b-4fd3-a980-2c5c02950f2b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8cb60118608742c1bfe3c0f93a9d99b9'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:29.793414 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7452246f-4b3a-4e09-ad26-9b04f450bb7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:29.793990 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7452246f-4b3a-4e09-ad26-9b04f450bb7f admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:29.794877 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7452246f-4b3a-4e09-ad26-9b04f450bb7f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:37:29.800144 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7452246f-4b3a-4e09-ad26-9b04f450bb7f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:29.805367 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7452246f-4b3a-4e09-ad26-9b04f450bb7f admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:29.807986 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-1ce867b0-771b-4fd3-a980-2c5c02950f2b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8cb60118608742c1bfe3c0f93a9d99b9'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:29.850996 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 963/1913] 10.222.0.99 () {58 vars in 1223 bytes} [Thu Jul 14 15:37:29 2022] DELETE /identity/v3/users/8cb60118608742c1bfe3c0f93a9d99b9 => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:37:29.856123 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 951/1914] 10.222.0.99 () {58 vars in 1152 bytes} [Thu Jul 14 15:37:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:37:29.860896 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce867b0-771b-4fd3-a980-2c5c02950f2b admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:29.862535 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-1ce867b0-771b-4fd3-a980-2c5c02950f2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:29.942401 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:37:30.026266 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7452246f-4b3a-4e09-ad26-9b04f450bb7f admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:30.028216 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-7452246f-4b3a-4e09-ad26-9b04f450bb7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:30.031743 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce867b0-771b-4fd3-a980-2c5c02950f2b admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:30.036873 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adb99ba-7ee6-49c7-bc32-915b7eb04c47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:30.038107 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adb99ba-7ee6-49c7-bc32-915b7eb04c47 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:30.038553 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adb99ba-7ee6-49c7-bc32-915b7eb04c47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:30.038989 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adb99ba-7ee6-49c7-bc32-915b7eb04c47 admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:30.039948 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adb99ba-7ee6-49c7-bc32-915b7eb04c47 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:30.042597 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adb99ba-7ee6-49c7-bc32-915b7eb04c47 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:30.051029 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7452246f-4b3a-4e09-ad26-9b04f450bb7f admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:30.057478 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8ff3c2-3f8b-439f-94b4-eb915c8fe310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:30.058745 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8ff3c2-3f8b-439f-94b4-eb915c8fe310 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:30.060677 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8ff3c2-3f8b-439f-94b4-eb915c8fe310 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:30.061978 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8ff3c2-3f8b-439f-94b4-eb915c8fe310 admin admin] PATH_INFO: `/v3/users/47c16ba599394d9cb3e0261f1e89cc80` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:30.064487 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8ff3c2-3f8b-439f-94b4-eb915c8fe310 admin admin] RBAC: Authorizing `identity:delete_user(user_id=47c16ba599394d9cb3e0261f1e89cc80)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:30.068304 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8ff3c2-3f8b-439f-94b4-eb915c8fe310 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:30.085077 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 964/1915] 10.222.0.99 () {60 vars in 1152 bytes} [Thu Jul 14 15:37:29 2022] POST /identity/v3/projects => generated 375 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:37:30.086461 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adb99ba-7ee6-49c7-bc32-915b7eb04c47 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:30.146128 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-6d8ff3c2-3f8b-439f-94b4-eb915c8fe310 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47c16ba599394d9cb3e0261f1e89cc80'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:30.168370 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-6d8ff3c2-3f8b-439f-94b4-eb915c8fe310 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47c16ba599394d9cb3e0261f1e89cc80'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:30.189041 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-6d8ff3c2-3f8b-439f-94b4-eb915c8fe310 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47c16ba599394d9cb3e0261f1e89cc80'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:30.190323 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-6d8ff3c2-3f8b-439f-94b4-eb915c8fe310 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47c16ba599394d9cb3e0261f1e89cc80'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:30.190460 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adb99ba-7ee6-49c7-bc32-915b7eb04c47 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:30.190662 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-6d8ff3c2-3f8b-439f-94b4-eb915c8fe310 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47c16ba599394d9cb3e0261f1e89cc80'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:30.196277 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8a2966-3298-458a-9871-5d14e69e1191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:30.197071 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8a2966-3298-458a-9871-5d14e69e1191 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:30.197468 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8a2966-3298-458a-9871-5d14e69e1191 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:30.197862 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8a2966-3298-458a-9871-5d14e69e1191 admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:30.198967 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8a2966-3298-458a-9871-5d14e69e1191 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:30.201175 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8a2966-3298-458a-9871-5d14e69e1191 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:30.206191 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 952/1916] 10.222.0.99 () {58 vars in 1223 bytes} [Thu Jul 14 15:37:30 2022] DELETE /identity/v3/users/47c16ba599394d9cb3e0261f1e89cc80 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:37:30.212177 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f0120d-efc9-453e-bc84-52828c285c53 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:30.212612 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f0120d-efc9-453e-bc84-52828c285c53 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:30.213004 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f0120d-efc9-453e-bc84-52828c285c53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:30.252517 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 965/1917] 10.222.0.99 () {60 vars in 1143 bytes} [Thu Jul 14 15:37:30 2022] POST /identity/v3/users => generated 288 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:37:30.256645 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8a2966-3298-458a-9871-5d14e69e1191 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:30.257245 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-b9f0120d-efc9-453e-bc84-52828c285c53 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:37:30.258375 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-1d8a2966-3298-458a-9871-5d14e69e1191 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:30.258523 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b9f0120d-efc9-453e-bc84-52828c285c53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:30.275354 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8a2966-3298-458a-9871-5d14e69e1191 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:30.296594 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:37:30.334783 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 953/1918] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:37:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:37:30.345731 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0f5474-09a8-4b7c-98ff-2b9f1e56f596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:30.346384 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f0120d-efc9-453e-bc84-52828c285c53 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:30.346629 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0f5474-09a8-4b7c-98ff-2b9f1e56f596 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:30.347061 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0f5474-09a8-4b7c-98ff-2b9f1e56f596 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:30.347394 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0f5474-09a8-4b7c-98ff-2b9f1e56f596 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:30.347802 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b9f0120d-efc9-453e-bc84-52828c285c53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:30.348505 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0f5474-09a8-4b7c-98ff-2b9f1e56f596 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:30.350574 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0f5474-09a8-4b7c-98ff-2b9f1e56f596 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:30.365198 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 966/1919] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:37:30 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 15:37:30.366314 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0f5474-09a8-4b7c-98ff-2b9f1e56f596 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:30.377563 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0f5474-09a8-4b7c-98ff-2b9f1e56f596 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:30.384476 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2963c5f-5807-46a9-acf9-2726118e4bfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:30.385164 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2963c5f-5807-46a9-acf9-2726118e4bfe admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:30.385486 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2963c5f-5807-46a9-acf9-2726118e4bfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:30.385773 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2963c5f-5807-46a9-acf9-2726118e4bfe admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:30.386515 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2963c5f-5807-46a9-acf9-2726118e4bfe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:30.388167 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2963c5f-5807-46a9-acf9-2726118e4bfe admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:30.404244 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 967/1920] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:37:30 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:37:30.404244 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2963c5f-5807-46a9-acf9-2726118e4bfe admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:30.404244 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-d2963c5f-5807-46a9-acf9-2726118e4bfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:30.417217 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2963c5f-5807-46a9-acf9-2726118e4bfe admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:30.420490 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f0120d-efc9-453e-bc84-52828c285c53 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:30.423502 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c623c5-05d2-4209-9a68-25e0ea72f25a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:30.424612 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c623c5-05d2-4209-9a68-25e0ea72f25a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:30.424878 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c704342-ade3-44b8-8e2c-0b89081c98ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:30.424996 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c623c5-05d2-4209-9a68-25e0ea72f25a admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:30.425332 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c623c5-05d2-4209-9a68-25e0ea72f25a admin admin] PATH_INFO: `/v3/projects/e3fb9d7add7544b0b07be3e2598eab4c` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:30.425695 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c704342-ade3-44b8-8e2c-0b89081c98ee service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:30.426051 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c704342-ade3-44b8-8e2c-0b89081c98ee service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:30.426538 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c623c5-05d2-4209-9a68-25e0ea72f25a admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3fb9d7add7544b0b07be3e2598eab4c)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:30.426966 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c704342-ade3-44b8-8e2c-0b89081c98ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:30.428500 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c623c5-05d2-4209-9a68-25e0ea72f25a admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:30.437294 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c704342-ade3-44b8-8e2c-0b89081c98ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:30.438776 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c704342-ade3-44b8-8e2c-0b89081c98ee service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:30.448912 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 954/1921] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:37:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 15:37:30.449634 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c704342-ade3-44b8-8e2c-0b89081c98ee service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:30.458255 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:37:30.459771 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c704342-ade3-44b8-8e2c-0b89081c98ee service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:30.465640 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9588fa1-432f-45bd-9f87-17ead4352f40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:30.466694 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9588fa1-432f-45bd-9f87-17ead4352f40 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:30.467037 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9588fa1-432f-45bd-9f87-17ead4352f40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:30.467330 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9588fa1-432f-45bd-9f87-17ead4352f40 admin admin] PATH_INFO: `/v3/projects/e338799da18942aa89350a3b4cfba6a9/users/cf782fc2f7dd418a80d94b73532106b9/roles/b8c98fddc0fd4436b577fd1d3361bc38` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:30.484911 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9588fa1-432f-45bd-9f87-17ead4352f40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e338799da18942aa89350a3b4cfba6a9, user_id=cf782fc2f7dd418a80d94b73532106b9, role_id=b8c98fddc0fd4436b577fd1d3361bc38)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:30.488097 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9588fa1-432f-45bd-9f87-17ead4352f40 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:30.490636 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-35c623c5-05d2-4209-9a68-25e0ea72f25a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3fb9d7add7544b0b07be3e2598eab4c', 'request_id': 'req-35c623c5-05d2-4209-9a68-25e0ea72f25a'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:30.500126 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-35c623c5-05d2-4209-9a68-25e0ea72f25a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3fb9d7add7544b0b07be3e2598eab4c', 'request_id': 'req-35c623c5-05d2-4209-9a68-25e0ea72f25a'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:30.507527 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 955/1922] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 15:37:30 2022] PUT /identity/v3/projects/e338799da18942aa89350a3b4cfba6a9/users/cf782fc2f7dd418a80d94b73532106b9/roles/b8c98fddc0fd4436b577fd1d3361bc38 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:37:30.511693 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-35c623c5-05d2-4209-9a68-25e0ea72f25a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3fb9d7add7544b0b07be3e2598eab4c', 'request_id': 'req-35c623c5-05d2-4209-9a68-25e0ea72f25a'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:30.519249 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9503d7-4a37-4eb5-812c-c4f732530f33 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:30.519709 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9503d7-4a37-4eb5-812c-c4f732530f33 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:30.520095 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9503d7-4a37-4eb5-812c-c4f732530f33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:30.531998 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-35c623c5-05d2-4209-9a68-25e0ea72f25a admin admin] The token cache is being invalidate because project e3fb9d7add7544b0b07be3e2598eab4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 15:37:30.532380 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-35c623c5-05d2-4209-9a68-25e0ea72f25a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:30.533168 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-35c623c5-05d2-4209-9a68-25e0ea72f25a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:30.534884 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 968/1923] 10.222.0.99 () {58 vars in 1232 bytes} [Thu Jul 14 15:37:30 2022] DELETE /identity/v3/projects/e3fb9d7add7544b0b07be3e2598eab4c => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:37:30.539771 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc3dd46-6b13-40bc-8fae-16c1ffd71a35 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:30.540113 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc3dd46-6b13-40bc-8fae-16c1ffd71a35 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:30.541126 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc3dd46-6b13-40bc-8fae-16c1ffd71a35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:30.554955 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-fd9503d7-4a37-4eb5-812c-c4f732530f33 None None] MFA Rules not processed for user `cf782fc2f7dd418a80d94b73532106b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:37:30.556595 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-fd9503d7-4a37-4eb5-812c-c4f732530f33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:30.562576 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-1cc3dd46-6b13-40bc-8fae-16c1ffd71a35 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:37:30.565353 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-1cc3dd46-6b13-40bc-8fae-16c1ffd71a35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:30.637913 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 956/1924] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:37:30 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:37:30.638488 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 969/1925] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:37:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 15:37:30.642648 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9503d7-4a37-4eb5-812c-c4f732530f33 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:30.644123 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-fd9503d7-4a37-4eb5-812c-c4f732530f33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:30.649027 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24afbe77-a51e-4e2a-bd43-4a49a25094e5 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:30.649438 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24afbe77-a51e-4e2a-bd43-4a49a25094e5 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:30.649801 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24afbe77-a51e-4e2a-bd43-4a49a25094e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:30.660478 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9503d7-4a37-4eb5-812c-c4f732530f33 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:30.665413 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57649469-acd3-4e3b-9072-a2657b340f21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:30.666157 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57649469-acd3-4e3b-9072-a2657b340f21 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:30.666507 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57649469-acd3-4e3b-9072-a2657b340f21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:30.666930 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57649469-acd3-4e3b-9072-a2657b340f21 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:30.667741 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57649469-acd3-4e3b-9072-a2657b340f21 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:30.671367 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57649469-acd3-4e3b-9072-a2657b340f21 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:30.672126 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-24afbe77-a51e-4e2a-bd43-4a49a25094e5 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:37:30.673444 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-24afbe77-a51e-4e2a-bd43-4a49a25094e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:30.683071 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 970/1926] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:37:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 15:37:30.688016 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24afbe77-a51e-4e2a-bd43-4a49a25094e5 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:30.699352 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24afbe77-a51e-4e2a-bd43-4a49a25094e5 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:30.706406 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 957/1927] 10.222.0.99 () {60 vars in 1152 bytes} [Thu Jul 14 15:37:30 2022] POST /identity/v3/projects => generated 375 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 15:37:30.706566 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9fc001-5f71-4b03-82af-8714b1f1f389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:30.706566 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9fc001-5f71-4b03-82af-8714b1f1f389 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:30.706566 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9fc001-5f71-4b03-82af-8714b1f1f389 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:30.706566 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9fc001-5f71-4b03-82af-8714b1f1f389 admin admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:30.707368 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9fc001-5f71-4b03-82af-8714b1f1f389 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:37:30.707502 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57649469-acd3-4e3b-9072-a2657b340f21 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:30.707768 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9fc001-5f71-4b03-82af-8714b1f1f389 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:30.709359 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9fc001-5f71-4b03-82af-8714b1f1f389 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:30.726425 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 971/1928] 10.222.0.99 () {58 vars in 1152 bytes} [Thu Jul 14 15:37:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:37:30.727767 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9fc001-5f71-4b03-82af-8714b1f1f389 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:30.741534 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9fc001-5f71-4b03-82af-8714b1f1f389 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:30.747038 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:37:30.805841 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57649469-acd3-4e3b-9072-a2657b340f21 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:30.810871 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431367c5-3b3f-4dce-bcad-7c4578a94717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:30.812206 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431367c5-3b3f-4dce-bcad-7c4578a94717 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:30.812623 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431367c5-3b3f-4dce-bcad-7c4578a94717 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:30.812922 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431367c5-3b3f-4dce-bcad-7c4578a94717 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:30.813627 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431367c5-3b3f-4dce-bcad-7c4578a94717 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:30.815569 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431367c5-3b3f-4dce-bcad-7c4578a94717 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:30.816493 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2bd597-488f-4adb-9cee-43c73e3fd021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:30.817199 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2bd597-488f-4adb-9cee-43c73e3fd021 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:30.817492 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2bd597-488f-4adb-9cee-43c73e3fd021 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:30.817773 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2bd597-488f-4adb-9cee-43c73e3fd021 admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:30.818475 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2bd597-488f-4adb-9cee-43c73e3fd021 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:30.820412 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2bd597-488f-4adb-9cee-43c73e3fd021 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:30.838570 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 958/1929] 10.222.0.99 () {60 vars in 1143 bytes} [Thu Jul 14 15:37:30 2022] POST /identity/v3/users => generated 294 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 15:37:30.839538 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431367c5-3b3f-4dce-bcad-7c4578a94717 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:30.845276 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-431367c5-3b3f-4dce-bcad-7c4578a94717 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:30.861908 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431367c5-3b3f-4dce-bcad-7c4578a94717 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:30.865906 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 972/1930] 10.222.0.99 () {60 vars in 1152 bytes} [Thu Jul 14 15:37:30 2022] POST /identity/v3/projects => generated 373 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:37:30.865906 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2bd597-488f-4adb-9cee-43c73e3fd021 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:30.888216 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:DELETE=7,keystone:SELECT=91,keystone:INSERT=2 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:37:30.936207 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a14bd95-fc58-4f04-9e1c-7e7f29bb14c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:30.939810 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a14bd95-fc58-4f04-9e1c-7e7f29bb14c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:30.940276 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a14bd95-fc58-4f04-9e1c-7e7f29bb14c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:30.940708 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a14bd95-fc58-4f04-9e1c-7e7f29bb14c8 admin admin] PATH_INFO: `/v3/projects/7e3a8ab2527749e79988c5e02b1ae864` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:30.942451 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a14bd95-fc58-4f04-9e1c-7e7f29bb14c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e3a8ab2527749e79988c5e02b1ae864)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:30.949234 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2bd597-488f-4adb-9cee-43c73e3fd021 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:30.949578 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a14bd95-fc58-4f04-9e1c-7e7f29bb14c8 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:30.956769 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002d7926-01a5-48ea-8773-f2ec6ed248b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:30.957714 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002d7926-01a5-48ea-8773-f2ec6ed248b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:30.958433 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002d7926-01a5-48ea-8773-f2ec6ed248b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:30.959054 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002d7926-01a5-48ea-8773-f2ec6ed248b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:30.960453 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002d7926-01a5-48ea-8773-f2ec6ed248b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:30.962245 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002d7926-01a5-48ea-8773-f2ec6ed248b4 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:30.976951 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 973/1931] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:37:30 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:37:30.977997 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002d7926-01a5-48ea-8773-f2ec6ed248b4 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:31.004533 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002d7926-01a5-48ea-8773-f2ec6ed248b4 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:31.009608 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ef8f8c-8e65-4f08-aab6-5cde234c8b67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:31.010286 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ef8f8c-8e65-4f08-aab6-5cde234c8b67 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:31.010576 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ef8f8c-8e65-4f08-aab6-5cde234c8b67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:31.010860 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ef8f8c-8e65-4f08-aab6-5cde234c8b67 admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:31.011570 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ef8f8c-8e65-4f08-aab6-5cde234c8b67 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:31.013579 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ef8f8c-8e65-4f08-aab6-5cde234c8b67 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:31.043173 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 974/1932] 10.222.0.99 () {60 vars in 1143 bytes} [Thu Jul 14 15:37:30 2022] POST /identity/v3/users => generated 287 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:37:31.045020 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ef8f8c-8e65-4f08-aab6-5cde234c8b67 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:31.052349 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2a14bd95-fc58-4f04-9e1c-7e7f29bb14c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e3a8ab2527749e79988c5e02b1ae864', 'request_id': 'req-2a14bd95-fc58-4f04-9e1c-7e7f29bb14c8'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:31.058226 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ef8f8c-8e65-4f08-aab6-5cde234c8b67 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:31.061165 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2a14bd95-fc58-4f04-9e1c-7e7f29bb14c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e3a8ab2527749e79988c5e02b1ae864', 'request_id': 'req-2a14bd95-fc58-4f04-9e1c-7e7f29bb14c8'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:31.068273 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2a14bd95-fc58-4f04-9e1c-7e7f29bb14c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e3a8ab2527749e79988c5e02b1ae864', 'request_id': 'req-2a14bd95-fc58-4f04-9e1c-7e7f29bb14c8'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:31.086272 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2a14bd95-fc58-4f04-9e1c-7e7f29bb14c8 admin admin] The token cache is being invalidate because project 7e3a8ab2527749e79988c5e02b1ae864 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 15:37:31.086574 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2a14bd95-fc58-4f04-9e1c-7e7f29bb14c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:31.087282 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2a14bd95-fc58-4f04-9e1c-7e7f29bb14c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:31.091670 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 959/1933] 10.222.0.99 () {58 vars in 1232 bytes} [Thu Jul 14 15:37:30 2022] DELETE /identity/v3/projects/7e3a8ab2527749e79988c5e02b1ae864 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:37:31.093694 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a14bd95-fc58-4f04-9e1c-7e7f29bb14c8 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:31.096072 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-2a14bd95-fc58-4f04-9e1c-7e7f29bb14c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:31.133857 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a45671-c000-405d-9c01-3ff55fcb1af5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:31.135423 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a45671-c000-405d-9c01-3ff55fcb1af5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:31.135746 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a45671-c000-405d-9c01-3ff55fcb1af5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:31.136032 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a45671-c000-405d-9c01-3ff55fcb1af5 admin admin] PATH_INFO: `/v3/projects/3f76f2008b7c42b3a9dc99e9274ea78f/users/e8b6ba6c235640338a3e1b526efcc632/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:31.154100 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:37:31.157907 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a45671-c000-405d-9c01-3ff55fcb1af5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f76f2008b7c42b3a9dc99e9274ea78f, user_id=e8b6ba6c235640338a3e1b526efcc632, role_id=d0f6ba1314f14d9f9c816bdb5d0c3aac)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:31.161158 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a45671-c000-405d-9c01-3ff55fcb1af5 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:31.180187 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 975/1934] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 15:37:31 2022] PUT /identity/v3/projects/3f76f2008b7c42b3a9dc99e9274ea78f/users/e8b6ba6c235640338a3e1b526efcc632/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:37:31.184967 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a45671-c000-405d-9c01-3ff55fcb1af5 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:31.187449 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-42a45671-c000-405d-9c01-3ff55fcb1af5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:31.191529 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a14bd95-fc58-4f04-9e1c-7e7f29bb14c8 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:31.276897 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a45671-c000-405d-9c01-3ff55fcb1af5 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:31.286743 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67b8360-b2ef-4881-a627-cbddb75f04fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:31.288109 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67b8360-b2ef-4881-a627-cbddb75f04fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:31.288503 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67b8360-b2ef-4881-a627-cbddb75f04fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:31.288833 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67b8360-b2ef-4881-a627-cbddb75f04fa admin admin] PATH_INFO: `/v3/users/6e524b0e56fd48458a8733f68d50e6a5` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:31.290294 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb34b3f6-dfb6-468a-a7ba-b9c55f65578d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:31.290711 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67b8360-b2ef-4881-a627-cbddb75f04fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e524b0e56fd48458a8733f68d50e6a5)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:31.291223 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb34b3f6-dfb6-468a-a7ba-b9c55f65578d admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:31.291628 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb34b3f6-dfb6-468a-a7ba-b9c55f65578d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:31.292029 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb34b3f6-dfb6-468a-a7ba-b9c55f65578d admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:31.292934 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb34b3f6-dfb6-468a-a7ba-b9c55f65578d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:31.294978 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb34b3f6-dfb6-468a-a7ba-b9c55f65578d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:31.298569 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67b8360-b2ef-4881-a627-cbddb75f04fa admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:31.311150 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 960/1935] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:37:31 2022] GET /identity/v3/roles => generated 1420 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:37:31.312667 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb34b3f6-dfb6-468a-a7ba-b9c55f65578d admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:31.317237 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-eb34b3f6-dfb6-468a-a7ba-b9c55f65578d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:31.332275 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb34b3f6-dfb6-468a-a7ba-b9c55f65578d admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:31.339477 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=94,keystone:INSERT=4,keystone:DELETE=2 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:37:31.340692 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d9c990-7d96-4a3b-b004-087d032de074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:31.341660 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d9c990-7d96-4a3b-b004-087d032de074 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:31.341834 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d9c990-7d96-4a3b-b004-087d032de074 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:31.342157 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d9c990-7d96-4a3b-b004-087d032de074 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:31.343881 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d9c990-7d96-4a3b-b004-087d032de074 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:31.345560 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d9c990-7d96-4a3b-b004-087d032de074 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:31.361630 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 961/1936] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:37:31 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:37:31.362018 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d9c990-7d96-4a3b-b004-087d032de074 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:31.394348 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d9c990-7d96-4a3b-b004-087d032de074 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:31.399808 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-a67b8360-b2ef-4881-a627-cbddb75f04fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e524b0e56fd48458a8733f68d50e6a5'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:31.402115 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc1ad1c-2cc5-4997-b9f2-f8f6182fdcc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:31.403026 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc1ad1c-2cc5-4997-b9f2-f8f6182fdcc6 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:31.403468 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc1ad1c-2cc5-4997-b9f2-f8f6182fdcc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:31.403965 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc1ad1c-2cc5-4997-b9f2-f8f6182fdcc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:31.404992 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc1ad1c-2cc5-4997-b9f2-f8f6182fdcc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:31.407615 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc1ad1c-2cc5-4997-b9f2-f8f6182fdcc6 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:31.411423 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-a67b8360-b2ef-4881-a627-cbddb75f04fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e524b0e56fd48458a8733f68d50e6a5'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:31.426908 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 962/1937] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:37:31 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:37:31.428030 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc1ad1c-2cc5-4997-b9f2-f8f6182fdcc6 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:31.428425 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-a67b8360-b2ef-4881-a627-cbddb75f04fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e524b0e56fd48458a8733f68d50e6a5'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:31.437494 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-a67b8360-b2ef-4881-a627-cbddb75f04fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e524b0e56fd48458a8733f68d50e6a5'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:31.443836 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-a67b8360-b2ef-4881-a627-cbddb75f04fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e524b0e56fd48458a8733f68d50e6a5'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:31.443938 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc1ad1c-2cc5-4997-b9f2-f8f6182fdcc6 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:31.455831 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b9362d-a422-4819-b933-2b461d757da6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:31.459508 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 976/1938] 10.222.0.99 () {58 vars in 1223 bytes} [Thu Jul 14 15:37:31 2022] DELETE /identity/v3/users/6e524b0e56fd48458a8733f68d50e6a5 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:37:31.459508 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67b8360-b2ef-4881-a627-cbddb75f04fa admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:31.460578 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b9362d-a422-4819-b933-2b461d757da6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:31.461028 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b9362d-a422-4819-b933-2b461d757da6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:31.461464 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b9362d-a422-4819-b933-2b461d757da6 admin admin] PATH_INFO: `/v3/projects/3f76f2008b7c42b3a9dc99e9274ea78f/users/e8b6ba6c235640338a3e1b526efcc632/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:31.464154 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b9362d-a422-4819-b933-2b461d757da6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f76f2008b7c42b3a9dc99e9274ea78f, user_id=e8b6ba6c235640338a3e1b526efcc632, role_id=d0f6ba1314f14d9f9c816bdb5d0c3aac)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:31.469254 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b9362d-a422-4819-b933-2b461d757da6 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:31.471153 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-a67b8360-b2ef-4881-a627-cbddb75f04fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:31.488030 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 963/1939] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 15:37:31 2022] PUT /identity/v3/projects/3f76f2008b7c42b3a9dc99e9274ea78f/users/e8b6ba6c235640338a3e1b526efcc632/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:37:31.496297 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c944811-32cf-4651-a00a-cc647296995f None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:31.496618 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c944811-32cf-4651-a00a-cc647296995f None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:31.496805 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c944811-32cf-4651-a00a-cc647296995f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:31.530146 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-7c944811-32cf-4651-a00a-cc647296995f None None] MFA Rules not processed for user `e8b6ba6c235640338a3e1b526efcc632`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:37:31.532375 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-7c944811-32cf-4651-a00a-cc647296995f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:31.575702 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67b8360-b2ef-4881-a627-cbddb75f04fa admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:31.619251 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 964/1940] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:37:31 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:37:31.632161 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c944811-32cf-4651-a00a-cc647296995f None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:31.634117 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-7c944811-32cf-4651-a00a-cc647296995f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:31.663890 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee89b46e-c2c3-408c-b245-492350bf81ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:31.664821 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee89b46e-c2c3-408c-b245-492350bf81ba admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:31.665108 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee89b46e-c2c3-408c-b245-492350bf81ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:31.665393 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee89b46e-c2c3-408c-b245-492350bf81ba admin admin] PATH_INFO: `/v3/projects/ead3dc4d1dcf450098946be77432bce2/users/b04c8de464b14622ba4a4a3b5e7c3658/roles/b8c98fddc0fd4436b577fd1d3361bc38` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:31.684895 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee89b46e-c2c3-408c-b245-492350bf81ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=ead3dc4d1dcf450098946be77432bce2, user_id=b04c8de464b14622ba4a4a3b5e7c3658, role_id=b8c98fddc0fd4436b577fd1d3361bc38)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:31.688425 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee89b46e-c2c3-408c-b245-492350bf81ba admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:31.694193 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:37:31.707005 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 977/1941] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 15:37:31 2022] PUT /identity/v3/projects/ead3dc4d1dcf450098946be77432bce2/users/b04c8de464b14622ba4a4a3b5e7c3658/roles/b8c98fddc0fd4436b577fd1d3361bc38 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:37:31.712823 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802e0d7f-ef7b-4554-91a3-bda3c0509f14 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:31.713119 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802e0d7f-ef7b-4554-91a3-bda3c0509f14 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:31.713406 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802e0d7f-ef7b-4554-91a3-bda3c0509f14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:31.722986 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c944811-32cf-4651-a00a-cc647296995f None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:31.732509 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-802e0d7f-ef7b-4554-91a3-bda3c0509f14 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:37:31.734515 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-802e0d7f-ef7b-4554-91a3-bda3c0509f14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:31.760872 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:DELETE=8,keystone:SELECT=90,keystone:INSERT=2 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:37:31.786091 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d9e91b-8bc6-46c0-a09a-ab7b4904bc55 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3e1a2b83f3d4d88ba5084dfaffd22f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:31.786899 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d9e91b-8bc6-46c0-a09a-ab7b4904bc55 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:31.787207 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d9e91b-8bc6-46c0-a09a-ab7b4904bc55 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:31.787616 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d9e91b-8bc6-46c0-a09a-ab7b4904bc55 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:31.789499 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f2d9e91b-8bc6-46c0-a09a-ab7b4904bc55 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:31.816008 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 978/1942] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:37:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:37:31.830338 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802e0d7f-ef7b-4554-91a3-bda3c0509f14 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:31.831983 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-802e0d7f-ef7b-4554-91a3-bda3c0509f14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:31.869068 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d9e91b-8bc6-46c0-a09a-ab7b4904bc55 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:31.870885 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d9e91b-8bc6-46c0-a09a-ab7b4904bc55 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:31.882049 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 965/1943] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:37:31 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:37:31.910551 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802e0d7f-ef7b-4554-91a3-bda3c0509f14 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:31.914681 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c6c561-7945-487c-89c7-ee953e73138e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:31.915525 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c6c561-7945-487c-89c7-ee953e73138e service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:31.915901 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c6c561-7945-487c-89c7-ee953e73138e service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:31.916242 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c6c561-7945-487c-89c7-ee953e73138e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:31.937830 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c6c561-7945-487c-89c7-ee953e73138e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:31.940230 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c6c561-7945-487c-89c7-ee953e73138e service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:31.954798 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 979/1944] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:37:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:37:32.050547 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d9e91b-8bc6-46c0-a09a-ab7b4904bc55 service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:32.051954 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f2d9e91b-8bc6-46c0-a09a-ab7b4904bc55 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:32.098974 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d9e91b-8bc6-46c0-a09a-ab7b4904bc55 service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:32.116122 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f492d265-c4ec-4950-8ecb-923c98cc2ee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:32.118264 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f492d265-c4ec-4950-8ecb-923c98cc2ee8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:32.118626 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f492d265-c4ec-4950-8ecb-923c98cc2ee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:32.119967 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f492d265-c4ec-4950-8ecb-923c98cc2ee8 admin admin] PATH_INFO: `/v3/projects/67b10df1450a43c5ab006b3872288e1b` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:32.122764 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f492d265-c4ec-4950-8ecb-923c98cc2ee8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=67b10df1450a43c5ab006b3872288e1b)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:32.129504 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f492d265-c4ec-4950-8ecb-923c98cc2ee8 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:32.213898 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-f492d265-c4ec-4950-8ecb-923c98cc2ee8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67b10df1450a43c5ab006b3872288e1b', 'request_id': 'req-f492d265-c4ec-4950-8ecb-923c98cc2ee8'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:32.226703 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-f492d265-c4ec-4950-8ecb-923c98cc2ee8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67b10df1450a43c5ab006b3872288e1b', 'request_id': 'req-f492d265-c4ec-4950-8ecb-923c98cc2ee8'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:32.238879 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-f492d265-c4ec-4950-8ecb-923c98cc2ee8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67b10df1450a43c5ab006b3872288e1b', 'request_id': 'req-f492d265-c4ec-4950-8ecb-923c98cc2ee8'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:32.268654 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-f492d265-c4ec-4950-8ecb-923c98cc2ee8 admin admin] The token cache is being invalidate because project 67b10df1450a43c5ab006b3872288e1b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 15:37:32.269069 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-f492d265-c4ec-4950-8ecb-923c98cc2ee8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:32.269866 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-f492d265-c4ec-4950-8ecb-923c98cc2ee8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:32.275926 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 966/1945] 10.222.0.99 () {58 vars in 1232 bytes} [Thu Jul 14 15:37:32 2022] DELETE /identity/v3/projects/67b10df1450a43c5ab006b3872288e1b => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:37:32.278272 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b41329-98de-441a-8ffe-807bf3856272 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:32.278567 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b41329-98de-441a-8ffe-807bf3856272 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:32.278912 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b41329-98de-441a-8ffe-807bf3856272 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:32.322192 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-a9b41329-98de-441a-8ffe-807bf3856272 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:37:32.325915 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-a9b41329-98de-441a-8ffe-807bf3856272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:32.486284 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a43c8b5-5b7b-485c-87d3-8c0be819ef47 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:32.486580 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a43c8b5-5b7b-485c-87d3-8c0be819ef47 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:32.487634 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a43c8b5-5b7b-485c-87d3-8c0be819ef47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:32.490202 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 980/1946] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:37:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:37:32.511120 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-5a43c8b5-5b7b-485c-87d3-8c0be819ef47 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:37:32.514275 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-5a43c8b5-5b7b-485c-87d3-8c0be819ef47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:32.557655 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 967/1947] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:37:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:37:32.564935 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b41329-98de-441a-8ffe-807bf3856272 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:32.575905 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b41329-98de-441a-8ffe-807bf3856272 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:32.582567 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38fd2fa-7403-457c-910f-a673ad1a7fc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:32.583101 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38fd2fa-7403-457c-910f-a673ad1a7fc4 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:32.583394 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38fd2fa-7403-457c-910f-a673ad1a7fc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:32.583835 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38fd2fa-7403-457c-910f-a673ad1a7fc4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:32.584684 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38fd2fa-7403-457c-910f-a673ad1a7fc4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:37:32.585006 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38fd2fa-7403-457c-910f-a673ad1a7fc4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:32.586830 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38fd2fa-7403-457c-910f-a673ad1a7fc4 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:32.602495 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 981/1948] 10.222.0.99 () {58 vars in 1152 bytes} [Thu Jul 14 15:37:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:37:32.606547 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a43c8b5-5b7b-485c-87d3-8c0be819ef47 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:32.618874 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a43c8b5-5b7b-485c-87d3-8c0be819ef47 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:32.625551 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09930c3c-e2ae-46c1-9efb-7f08c7639b2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:32.625986 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09930c3c-e2ae-46c1-9efb-7f08c7639b2c admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:32.626375 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09930c3c-e2ae-46c1-9efb-7f08c7639b2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:32.626733 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09930c3c-e2ae-46c1-9efb-7f08c7639b2c admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:32.627531 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09930c3c-e2ae-46c1-9efb-7f08c7639b2c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:32.632994 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09930c3c-e2ae-46c1-9efb-7f08c7639b2c admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:32.659683 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:37:32.676029 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 968/1949] 10.222.0.99 () {60 vars in 1152 bytes} [Thu Jul 14 15:37:32 2022] POST /identity/v3/projects => generated 375 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 15:37:32.678203 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38fd2fa-7403-457c-910f-a673ad1a7fc4 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:32.692036 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38fd2fa-7403-457c-910f-a673ad1a7fc4 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:32.758687 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:37:32.771701 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19229e75-694c-4e06-ae99-f935e8b89c7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:32.772771 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19229e75-694c-4e06-ae99-f935e8b89c7c admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:32.773157 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19229e75-694c-4e06-ae99-f935e8b89c7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:32.773651 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19229e75-694c-4e06-ae99-f935e8b89c7c admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:32.774653 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19229e75-694c-4e06-ae99-f935e8b89c7c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:32.779334 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19229e75-694c-4e06-ae99-f935e8b89c7c admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:32.805117 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 982/1950] 10.222.0.99 () {60 vars in 1143 bytes} [Thu Jul 14 15:37:32 2022] POST /identity/v3/users => generated 288 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 15:37:32.809922 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09930c3c-e2ae-46c1-9efb-7f08c7639b2c admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:32.822278 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09930c3c-e2ae-46c1-9efb-7f08c7639b2c admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:32.834296 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084394f5-1f65-4095-b4a5-cf4c41036328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:32.835111 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084394f5-1f65-4095-b4a5-cf4c41036328 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:32.835414 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084394f5-1f65-4095-b4a5-cf4c41036328 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:32.835732 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084394f5-1f65-4095-b4a5-cf4c41036328 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:32.836479 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084394f5-1f65-4095-b4a5-cf4c41036328 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:32.838134 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084394f5-1f65-4095-b4a5-cf4c41036328 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:32.851433 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 969/1951] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:37:32 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:37:32.855535 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19229e75-694c-4e06-ae99-f935e8b89c7c admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:32.867129 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19229e75-694c-4e06-ae99-f935e8b89c7c admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:32.871755 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b23176-d12b-4294-b571-b8508ec37b30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:32.872432 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b23176-d12b-4294-b571-b8508ec37b30 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:32.872720 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b23176-d12b-4294-b571-b8508ec37b30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:32.873035 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b23176-d12b-4294-b571-b8508ec37b30 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:32.874990 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b23176-d12b-4294-b571-b8508ec37b30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:32.876641 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b23176-d12b-4294-b571-b8508ec37b30 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:32.889987 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 983/1952] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:37:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:37:32.891669 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084394f5-1f65-4095-b4a5-cf4c41036328 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:32.902381 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084394f5-1f65-4095-b4a5-cf4c41036328 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:32.908676 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4785e60-5e61-4038-a827-9c9f62454742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:32.909639 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4785e60-5e61-4038-a827-9c9f62454742 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:32.909929 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4785e60-5e61-4038-a827-9c9f62454742 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:32.910200 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4785e60-5e61-4038-a827-9c9f62454742 admin admin] PATH_INFO: `/v3/projects/af7cfa8cb5af4b9a8eaf378d1362dacf/users/f1eb0a4d069940cdb2fa6d0c643bacfc/roles/b8c98fddc0fd4436b577fd1d3361bc38` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:32.928693 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4785e60-5e61-4038-a827-9c9f62454742 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af7cfa8cb5af4b9a8eaf378d1362dacf, user_id=f1eb0a4d069940cdb2fa6d0c643bacfc, role_id=b8c98fddc0fd4436b577fd1d3361bc38)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:32.931386 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4785e60-5e61-4038-a827-9c9f62454742 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:32.946829 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 970/1953] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 15:37:32 2022] PUT /identity/v3/projects/af7cfa8cb5af4b9a8eaf378d1362dacf/users/f1eb0a4d069940cdb2fa6d0c643bacfc/roles/b8c98fddc0fd4436b577fd1d3361bc38 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 15:37:32.958683 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23186207-9bd1-46be-b828-3c2a90e615c7 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:32.958978 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23186207-9bd1-46be-b828-3c2a90e615c7 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:32.959261 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23186207-9bd1-46be-b828-3c2a90e615c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:32.985123 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2379f39f-4f59-45eb-8bb7-a0756464bfe2 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:32.985123 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2379f39f-4f59-45eb-8bb7-a0756464bfe2 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:32.985447 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2379f39f-4f59-45eb-8bb7-a0756464bfe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:32.995380 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-23186207-9bd1-46be-b828-3c2a90e615c7 None None] MFA Rules not processed for user `f1eb0a4d069940cdb2fa6d0c643bacfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:37:32.997111 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-23186207-9bd1-46be-b828-3c2a90e615c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:33.029166 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-2379f39f-4f59-45eb-8bb7-a0756464bfe2 None None] MFA Rules not processed for user `b04c8de464b14622ba4a4a3b5e7c3658`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:37:33.030962 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-2379f39f-4f59-45eb-8bb7-a0756464bfe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:33.081415 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 984/1954] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:37:32 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:37:33.085924 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23186207-9bd1-46be-b828-3c2a90e615c7 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:33.122156 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 971/1955] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:37:32 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:37:33.123480 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2379f39f-4f59-45eb-8bb7-a0756464bfe2 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:33.125965 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-2379f39f-4f59-45eb-8bb7-a0756464bfe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:33.161119 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23186207-9bd1-46be-b828-3c2a90e615c7 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:33.168242 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97e2c0d-44c1-46e0-88d4-19a02dc25658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:33.169063 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97e2c0d-44c1-46e0-88d4-19a02dc25658 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:33.169431 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97e2c0d-44c1-46e0-88d4-19a02dc25658 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:33.169785 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97e2c0d-44c1-46e0-88d4-19a02dc25658 admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:33.171696 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97e2c0d-44c1-46e0-88d4-19a02dc25658 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:33.174465 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97e2c0d-44c1-46e0-88d4-19a02dc25658 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:33.200822 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2379f39f-4f59-45eb-8bb7-a0756464bfe2 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:33.202621 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 985/1956] 10.222.0.99 () {60 vars in 1152 bytes} [Thu Jul 14 15:37:33 2022] POST /identity/v3/projects => generated 379 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 15:37:33.206552 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97e2c0d-44c1-46e0-88d4-19a02dc25658 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:33.258452 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:37:33.263671 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bc2714-8721-4ed7-b76a-7db6b786cdfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:33.264345 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bc2714-8721-4ed7-b76a-7db6b786cdfb admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:33.264686 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bc2714-8721-4ed7-b76a-7db6b786cdfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:33.264955 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bc2714-8721-4ed7-b76a-7db6b786cdfb admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:33.265821 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bc2714-8721-4ed7-b76a-7db6b786cdfb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:33.267862 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bc2714-8721-4ed7-b76a-7db6b786cdfb admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:33.272227 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97e2c0d-44c1-46e0-88d4-19a02dc25658 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:33.273125 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:37:33.277371 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44851b80-820e-4631-b00f-c38c3fd97dd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:33.278473 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44851b80-820e-4631-b00f-c38c3fd97dd2 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:33.279130 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44851b80-820e-4631-b00f-c38c3fd97dd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:33.279725 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44851b80-820e-4631-b00f-c38c3fd97dd2 admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:33.281498 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44851b80-820e-4631-b00f-c38c3fd97dd2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:33.283553 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44851b80-820e-4631-b00f-c38c3fd97dd2 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:33.299128 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 972/1957] 10.222.0.99 () {60 vars in 1152 bytes} [Thu Jul 14 15:37:33 2022] POST /identity/v3/projects => generated 371 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:37:33.303114 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bc2714-8721-4ed7-b76a-7db6b786cdfb admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:33.309543 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 986/1958] 10.222.0.99 () {60 vars in 1143 bytes} [Thu Jul 14 15:37:33 2022] POST /identity/v3/users => generated 296 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 15:37:33.314375 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44851b80-820e-4631-b00f-c38c3fd97dd2 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:33.324012 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44851b80-820e-4631-b00f-c38c3fd97dd2 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:33.373042 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bc2714-8721-4ed7-b76a-7db6b786cdfb admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:33.379055 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4bc93d-c41d-404d-ac3f-aa1288ca22e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:33.379783 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4bc93d-c41d-404d-ac3f-aa1288ca22e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:33.380159 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4bc93d-c41d-404d-ac3f-aa1288ca22e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:33.380446 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4bc93d-c41d-404d-ac3f-aa1288ca22e2 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:33.381142 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4bc93d-c41d-404d-ac3f-aa1288ca22e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:33.383070 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4bc93d-c41d-404d-ac3f-aa1288ca22e2 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:33.398651 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3e7a68-20ca-4d6b-a6b8-966755eefc03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:33.399746 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3e7a68-20ca-4d6b-a6b8-966755eefc03 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:33.399990 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3e7a68-20ca-4d6b-a6b8-966755eefc03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:33.400272 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3e7a68-20ca-4d6b-a6b8-966755eefc03 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:33.401288 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3e7a68-20ca-4d6b-a6b8-966755eefc03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:33.403097 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3e7a68-20ca-4d6b-a6b8-966755eefc03 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:33.407454 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 973/1959] 10.222.0.99 () {60 vars in 1143 bytes} [Thu Jul 14 15:37:33 2022] POST /identity/v3/users => generated 292 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 15:37:33.411140 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4bc93d-c41d-404d-ac3f-aa1288ca22e2 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:33.414518 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 987/1960] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:37:33 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:37:33.418126 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3e7a68-20ca-4d6b-a6b8-966755eefc03 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:33.423323 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4bc93d-c41d-404d-ac3f-aa1288ca22e2 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:33.430588 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc0bad7-7951-4931-802f-5eb3bf4fc991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:33.431266 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc0bad7-7951-4931-802f-5eb3bf4fc991 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:33.431579 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc0bad7-7951-4931-802f-5eb3bf4fc991 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:33.431880 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc0bad7-7951-4931-802f-5eb3bf4fc991 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:33.432599 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc0bad7-7951-4931-802f-5eb3bf4fc991 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:33.434346 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc0bad7-7951-4931-802f-5eb3bf4fc991 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:33.434624 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3e7a68-20ca-4d6b-a6b8-966755eefc03 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:33.441593 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1aee590-e935-458d-a8fa-5f19fe12f057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:33.443711 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aee590-e935-458d-a8fa-5f19fe12f057 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:33.444079 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aee590-e935-458d-a8fa-5f19fe12f057 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:33.444438 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aee590-e935-458d-a8fa-5f19fe12f057 admin admin] PATH_INFO: `/v3/projects/7998b6c82ca948faacef764291158e7a/users/a7b2d8e27ee84e809b35779311b8caf2/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:33.451715 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 974/1961] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:37:33 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 15:37:33.453540 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc0bad7-7951-4931-802f-5eb3bf4fc991 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:33.465130 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc0bad7-7951-4931-802f-5eb3bf4fc991 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:33.466582 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1aee590-e935-458d-a8fa-5f19fe12f057 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7998b6c82ca948faacef764291158e7a, user_id=a7b2d8e27ee84e809b35779311b8caf2, role_id=d0f6ba1314f14d9f9c816bdb5d0c3aac)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:33.469679 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1aee590-e935-458d-a8fa-5f19fe12f057 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:33.475110 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057ae526-4084-4c87-a67e-09b86ee39f5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:33.476122 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057ae526-4084-4c87-a67e-09b86ee39f5e admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:33.476468 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057ae526-4084-4c87-a67e-09b86ee39f5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:33.476755 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057ae526-4084-4c87-a67e-09b86ee39f5e admin admin] PATH_INFO: `/v3/projects/a597de03a82a4bb090c829e1beb8eaa7/users/70a38ee462734c6ab5c91aa3fbac669c/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:33.486670 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 988/1962] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 15:37:33 2022] PUT /identity/v3/projects/7998b6c82ca948faacef764291158e7a/users/a7b2d8e27ee84e809b35779311b8caf2/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 15:37:33.490635 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1aee590-e935-458d-a8fa-5f19fe12f057 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:33.496815 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057ae526-4084-4c87-a67e-09b86ee39f5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a597de03a82a4bb090c829e1beb8eaa7, user_id=70a38ee462734c6ab5c91aa3fbac669c, role_id=d0f6ba1314f14d9f9c816bdb5d0c3aac)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:33.502331 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057ae526-4084-4c87-a67e-09b86ee39f5e admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:33.523637 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 975/1963] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 15:37:33 2022] PUT /identity/v3/projects/a597de03a82a4bb090c829e1beb8eaa7/users/70a38ee462734c6ab5c91aa3fbac669c/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:37:33.527695 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057ae526-4084-4c87-a67e-09b86ee39f5e admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:33.590385 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1aee590-e935-458d-a8fa-5f19fe12f057 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:33.614927 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057ae526-4084-4c87-a67e-09b86ee39f5e admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:33.620758 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d0ef75-9a13-40fe-b957-22782534684d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:33.621663 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d0ef75-9a13-40fe-b957-22782534684d admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:33.622254 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d0ef75-9a13-40fe-b957-22782534684d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:33.622677 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d0ef75-9a13-40fe-b957-22782534684d admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:33.623575 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d0ef75-9a13-40fe-b957-22782534684d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:33.626197 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d0ef75-9a13-40fe-b957-22782534684d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:33.648796 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 976/1964] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:37:33 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 15:37:33.652167 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d0ef75-9a13-40fe-b957-22782534684d admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:33.666091 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d0ef75-9a13-40fe-b957-22782534684d admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:33.673034 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51839cf2-ad06-47ed-a3ba-21463a0e26dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:33.674216 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51839cf2-ad06-47ed-a3ba-21463a0e26dd admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:33.674513 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51839cf2-ad06-47ed-a3ba-21463a0e26dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:33.674970 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51839cf2-ad06-47ed-a3ba-21463a0e26dd admin admin] PATH_INFO: `/v3/projects/a597de03a82a4bb090c829e1beb8eaa7/users/70a38ee462734c6ab5c91aa3fbac669c/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:33.677842 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51839cf2-ad06-47ed-a3ba-21463a0e26dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=a597de03a82a4bb090c829e1beb8eaa7, user_id=70a38ee462734c6ab5c91aa3fbac669c, role_id=d0f6ba1314f14d9f9c816bdb5d0c3aac)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:33.681838 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51839cf2-ad06-47ed-a3ba-21463a0e26dd admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:33.685294 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0470b59-ef35-4039-b56b-563e9414314c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:33.688130 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0470b59-ef35-4039-b56b-563e9414314c admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:33.688666 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0470b59-ef35-4039-b56b-563e9414314c admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:33.689235 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0470b59-ef35-4039-b56b-563e9414314c admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:33.690273 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0470b59-ef35-4039-b56b-563e9414314c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:33.693404 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0470b59-ef35-4039-b56b-563e9414314c admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:33.694966 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:37:33.704368 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 977/1965] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 15:37:33 2022] PUT /identity/v3/projects/a597de03a82a4bb090c829e1beb8eaa7/users/70a38ee462734c6ab5c91aa3fbac669c/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:37:33.715275 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 989/1966] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:37:33 2022] GET /identity/v3/roles => generated 1420 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:37:33.720069 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51839cf2-ad06-47ed-a3ba-21463a0e26dd admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:33.778062 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:37:33.832027 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51839cf2-ad06-47ed-a3ba-21463a0e26dd admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:33.839290 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba28624-a24a-4407-884d-7f6e1b3bd940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:33.840592 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba28624-a24a-4407-884d-7f6e1b3bd940 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:33.841053 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba28624-a24a-4407-884d-7f6e1b3bd940 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:33.841784 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba28624-a24a-4407-884d-7f6e1b3bd940 admin admin] PATH_INFO: `/v3/projects/7998b6c82ca948faacef764291158e7a/users/a7b2d8e27ee84e809b35779311b8caf2/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:33.846465 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba28624-a24a-4407-884d-7f6e1b3bd940 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7998b6c82ca948faacef764291158e7a, user_id=a7b2d8e27ee84e809b35779311b8caf2, role_id=d0f6ba1314f14d9f9c816bdb5d0c3aac)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:33.851627 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba28624-a24a-4407-884d-7f6e1b3bd940 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:33.866293 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 978/1967] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 15:37:33 2022] PUT /identity/v3/projects/7998b6c82ca948faacef764291158e7a/users/a7b2d8e27ee84e809b35779311b8caf2/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 15:37:33.878336 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6af56a-1fe5-4925-ac04-3b3e97910a0e None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:33.878656 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6af56a-1fe5-4925-ac04-3b3e97910a0e None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:33.878937 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6af56a-1fe5-4925-ac04-3b3e97910a0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:33.909914 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-af6af56a-1fe5-4925-ac04-3b3e97910a0e None None] MFA Rules not processed for user `a7b2d8e27ee84e809b35779311b8caf2`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:37:33.911147 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-af6af56a-1fe5-4925-ac04-3b3e97910a0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:34.007200 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 990/1968] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:37:33 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 15:37:34.023622 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba28624-a24a-4407-884d-7f6e1b3bd940 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:34.025553 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-fba28624-a24a-4407-884d-7f6e1b3bd940 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:34.095254 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba28624-a24a-4407-884d-7f6e1b3bd940 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:34.098960 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3bd095-3bf9-447a-9453-ef42695006f3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3e1a2b83f3d4d88ba5084dfaffd22f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:34.099650 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3bd095-3bf9-447a-9453-ef42695006f3 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:34.099949 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3bd095-3bf9-447a-9453-ef42695006f3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:34.100233 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3bd095-3bf9-447a-9453-ef42695006f3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:34.110665 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3bd095-3bf9-447a-9453-ef42695006f3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:34.112226 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3bd095-3bf9-447a-9453-ef42695006f3 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:34.123160 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 979/1969] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:37:34 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:37:34.493911 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6af56a-1fe5-4925-ac04-3b3e97910a0e None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:34.503478 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6af56a-1fe5-4925-ac04-3b3e97910a0e None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:34.508092 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7179290a-3517-45d5-9fa3-23307c2394a8 tempest-VolumeQuotaClassesTest-911059298 tempest-VolumeQuotaClassesTest-911059298-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b2d8e27ee84e809b35779311b8caf2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7998b6c82ca948faacef764291158e7a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:34.508756 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7179290a-3517-45d5-9fa3-23307c2394a8 tempest-VolumeQuotaClassesTest-911059298 tempest-VolumeQuotaClassesTest-911059298-project-admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:34.509049 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7179290a-3517-45d5-9fa3-23307c2394a8 tempest-VolumeQuotaClassesTest-911059298 tempest-VolumeQuotaClassesTest-911059298-project-admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:34.509328 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7179290a-3517-45d5-9fa3-23307c2394a8 tempest-VolumeQuotaClassesTest-911059298 tempest-VolumeQuotaClassesTest-911059298-project-admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:34.510081 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7179290a-3517-45d5-9fa3-23307c2394a8 tempest-VolumeQuotaClassesTest-911059298 tempest-VolumeQuotaClassesTest-911059298-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:37:34.510386 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7179290a-3517-45d5-9fa3-23307c2394a8 tempest-VolumeQuotaClassesTest-911059298 tempest-VolumeQuotaClassesTest-911059298-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:34.512052 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7179290a-3517-45d5-9fa3-23307c2394a8 tempest-VolumeQuotaClassesTest-911059298 tempest-VolumeQuotaClassesTest-911059298-project-admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:34.532749 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 991/1970] 10.222.0.99 () {58 vars in 1152 bytes} [Thu Jul 14 15:37:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:37:34.536950 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3bd095-3bf9-447a-9453-ef42695006f3 service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:34.546956 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3bd095-3bf9-447a-9453-ef42695006f3 service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:34.552533 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7863b926-07bd-40d5-a39f-d6f9a42a9a39 tempest-VolumeQuotaClassesTest-911059298 tempest-VolumeQuotaClassesTest-911059298-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b2d8e27ee84e809b35779311b8caf2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7998b6c82ca948faacef764291158e7a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:34.553244 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7863b926-07bd-40d5-a39f-d6f9a42a9a39 tempest-VolumeQuotaClassesTest-911059298 tempest-VolumeQuotaClassesTest-911059298-project-admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:34.553538 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7863b926-07bd-40d5-a39f-d6f9a42a9a39 tempest-VolumeQuotaClassesTest-911059298 tempest-VolumeQuotaClassesTest-911059298-project-admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:34.553832 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7863b926-07bd-40d5-a39f-d6f9a42a9a39 tempest-VolumeQuotaClassesTest-911059298 tempest-VolumeQuotaClassesTest-911059298-project-admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:34.554549 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7863b926-07bd-40d5-a39f-d6f9a42a9a39 tempest-VolumeQuotaClassesTest-911059298 tempest-VolumeQuotaClassesTest-911059298-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:34.556519 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7863b926-07bd-40d5-a39f-d6f9a42a9a39 tempest-VolumeQuotaClassesTest-911059298 tempest-VolumeQuotaClassesTest-911059298-project-admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:34.594717 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 980/1971] 10.222.0.99 () {60 vars in 1152 bytes} [Thu Jul 14 15:37:34 2022] POST /identity/v3/projects => generated 354 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 15:37:34.596197 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7179290a-3517-45d5-9fa3-23307c2394a8 tempest-VolumeQuotaClassesTest-911059298 tempest-VolumeQuotaClassesTest-911059298-project-admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:34.617638 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7179290a-3517-45d5-9fa3-23307c2394a8 tempest-VolumeQuotaClassesTest-911059298 tempest-VolumeQuotaClassesTest-911059298-project-admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:34.626667 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1b44bf-3a1c-44f3-a4d5-b2b0a11d057a None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:34.627042 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1b44bf-3a1c-44f3-a4d5-b2b0a11d057a None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:34.627393 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1b44bf-3a1c-44f3-a4d5-b2b0a11d057a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:34.688364 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-4e1b44bf-3a1c-44f3-a4d5-b2b0a11d057a None None] MFA Rules not processed for user `70a38ee462734c6ab5c91aa3fbac669c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:37:34.692805 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-4e1b44bf-3a1c-44f3-a4d5-b2b0a11d057a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:34.701489 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219e519d-4e77-425b-82c4-a0c04129064d tempest-VolumeQuotaClassesTest-911059298 tempest-VolumeQuotaClassesTest-911059298-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b2d8e27ee84e809b35779311b8caf2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7998b6c82ca948faacef764291158e7a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:34.703641 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219e519d-4e77-425b-82c4-a0c04129064d tempest-VolumeQuotaClassesTest-911059298 tempest-VolumeQuotaClassesTest-911059298-project-admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:34.704145 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219e519d-4e77-425b-82c4-a0c04129064d tempest-VolumeQuotaClassesTest-911059298 tempest-VolumeQuotaClassesTest-911059298-project-admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:34.705280 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219e519d-4e77-425b-82c4-a0c04129064d tempest-VolumeQuotaClassesTest-911059298 tempest-VolumeQuotaClassesTest-911059298-project-admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:34.706300 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219e519d-4e77-425b-82c4-a0c04129064d tempest-VolumeQuotaClassesTest-911059298 tempest-VolumeQuotaClassesTest-911059298-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:37:34.706836 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219e519d-4e77-425b-82c4-a0c04129064d tempest-VolumeQuotaClassesTest-911059298 tempest-VolumeQuotaClassesTest-911059298-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:34.708777 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219e519d-4e77-425b-82c4-a0c04129064d tempest-VolumeQuotaClassesTest-911059298 tempest-VolumeQuotaClassesTest-911059298-project-admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:34.728928 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 992/1972] 10.222.0.99 () {58 vars in 1152 bytes} [Thu Jul 14 15:37:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:37:34.777288 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219e519d-4e77-425b-82c4-a0c04129064d tempest-VolumeQuotaClassesTest-911059298 tempest-VolumeQuotaClassesTest-911059298-project-admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:34.792394 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219e519d-4e77-425b-82c4-a0c04129064d tempest-VolumeQuotaClassesTest-911059298 tempest-VolumeQuotaClassesTest-911059298-project-admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:34.801312 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac87cf5c-21e9-4501-ac36-861739d3bebe tempest-VolumeQuotaClassesTest-911059298 tempest-VolumeQuotaClassesTest-911059298-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7b2d8e27ee84e809b35779311b8caf2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7998b6c82ca948faacef764291158e7a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:34.803061 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac87cf5c-21e9-4501-ac36-861739d3bebe tempest-VolumeQuotaClassesTest-911059298 tempest-VolumeQuotaClassesTest-911059298-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:34.803510 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac87cf5c-21e9-4501-ac36-861739d3bebe tempest-VolumeQuotaClassesTest-911059298 tempest-VolumeQuotaClassesTest-911059298-project-admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:34.804103 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac87cf5c-21e9-4501-ac36-861739d3bebe tempest-VolumeQuotaClassesTest-911059298 tempest-VolumeQuotaClassesTest-911059298-project-admin] PATH_INFO: `/v3/projects/04613bea14f747fdb215b5ed211106e6` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:34.806161 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac87cf5c-21e9-4501-ac36-861739d3bebe tempest-VolumeQuotaClassesTest-911059298 tempest-VolumeQuotaClassesTest-911059298-project-admin] RBAC: Authorizing `identity:delete_project(project_id=04613bea14f747fdb215b5ed211106e6)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:34.809182 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac87cf5c-21e9-4501-ac36-861739d3bebe tempest-VolumeQuotaClassesTest-911059298 tempest-VolumeQuotaClassesTest-911059298-project-admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:34.871663 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 981/1973] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:37:34 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:37:34.883857 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-ac87cf5c-21e9-4501-ac36-861739d3bebe tempest-VolumeQuotaClassesTest-911059298 tempest-VolumeQuotaClassesTest-911059298-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04613bea14f747fdb215b5ed211106e6', 'request_id': 'req-ac87cf5c-21e9-4501-ac36-861739d3bebe'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:34.884048 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1b44bf-3a1c-44f3-a4d5-b2b0a11d057a None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:34.885631 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-4e1b44bf-3a1c-44f3-a4d5-b2b0a11d057a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:34.892620 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-ac87cf5c-21e9-4501-ac36-861739d3bebe tempest-VolumeQuotaClassesTest-911059298 tempest-VolumeQuotaClassesTest-911059298-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04613bea14f747fdb215b5ed211106e6', 'request_id': 'req-ac87cf5c-21e9-4501-ac36-861739d3bebe'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:34.895296 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:37:34.899514 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-ac87cf5c-21e9-4501-ac36-861739d3bebe tempest-VolumeQuotaClassesTest-911059298 tempest-VolumeQuotaClassesTest-911059298-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04613bea14f747fdb215b5ed211106e6', 'request_id': 'req-ac87cf5c-21e9-4501-ac36-861739d3bebe'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:34.926037 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-ac87cf5c-21e9-4501-ac36-861739d3bebe tempest-VolumeQuotaClassesTest-911059298 tempest-VolumeQuotaClassesTest-911059298-project-admin] The token cache is being invalidate because project 04613bea14f747fdb215b5ed211106e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 15:37:34.926335 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-ac87cf5c-21e9-4501-ac36-861739d3bebe tempest-VolumeQuotaClassesTest-911059298 tempest-VolumeQuotaClassesTest-911059298-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:34.927007 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-ac87cf5c-21e9-4501-ac36-861739d3bebe tempest-VolumeQuotaClassesTest-911059298 tempest-VolumeQuotaClassesTest-911059298-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:34.929074 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:37:34.929320 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 993/1974] 10.222.0.99 () {58 vars in 1232 bytes} [Thu Jul 14 15:37:34 2022] DELETE /identity/v3/projects/04613bea14f747fdb215b5ed211106e6 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 15:37:34.984064 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1b44bf-3a1c-44f3-a4d5-b2b0a11d057a None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:35.062005 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0505480e-5dc1-4ecd-b30f-fc4d4cb1df9c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3e1a2b83f3d4d88ba5084dfaffd22f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:35.062911 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0505480e-5dc1-4ecd-b30f-fc4d4cb1df9c service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:35.063357 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0505480e-5dc1-4ecd-b30f-fc4d4cb1df9c service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:35.063625 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0505480e-5dc1-4ecd-b30f-fc4d4cb1df9c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:35.065935 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0505480e-5dc1-4ecd-b30f-fc4d4cb1df9c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:35.161320 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0505480e-5dc1-4ecd-b30f-fc4d4cb1df9c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:35.163095 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0505480e-5dc1-4ecd-b30f-fc4d4cb1df9c service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:35.173688 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 982/1975] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:37:34 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 15:37:35.215628 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac87cf5c-21e9-4501-ac36-861739d3bebe tempest-VolumeQuotaClassesTest-911059298 tempest-VolumeQuotaClassesTest-911059298-project-admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:35.218017 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-ac87cf5c-21e9-4501-ac36-861739d3bebe tempest-VolumeQuotaClassesTest-911059298 tempest-VolumeQuotaClassesTest-911059298-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:35.295421 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac87cf5c-21e9-4501-ac36-861739d3bebe tempest-VolumeQuotaClassesTest-911059298 tempest-VolumeQuotaClassesTest-911059298-project-admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:35.301615 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a06e52-3ef8-464b-917b-51bbd5f33dc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:35.302514 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a06e52-3ef8-464b-917b-51bbd5f33dc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:35.303072 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a06e52-3ef8-464b-917b-51bbd5f33dc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:35.303072 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a06e52-3ef8-464b-917b-51bbd5f33dc7 admin admin] PATH_INFO: `/v3/users/f1eb0a4d069940cdb2fa6d0c643bacfc` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:35.304381 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a06e52-3ef8-464b-917b-51bbd5f33dc7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1eb0a4d069940cdb2fa6d0c643bacfc)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:35.306583 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a06e52-3ef8-464b-917b-51bbd5f33dc7 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:35.370136 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-a6a06e52-3ef8-464b-917b-51bbd5f33dc7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1eb0a4d069940cdb2fa6d0c643bacfc'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:35.381000 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-a6a06e52-3ef8-464b-917b-51bbd5f33dc7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1eb0a4d069940cdb2fa6d0c643bacfc'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:35.407983 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-a6a06e52-3ef8-464b-917b-51bbd5f33dc7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1eb0a4d069940cdb2fa6d0c643bacfc'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:35.408842 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-a6a06e52-3ef8-464b-917b-51bbd5f33dc7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1eb0a4d069940cdb2fa6d0c643bacfc'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:35.409557 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-a6a06e52-3ef8-464b-917b-51bbd5f33dc7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1eb0a4d069940cdb2fa6d0c643bacfc'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:35.423525 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 994/1976] 10.222.0.99 () {58 vars in 1223 bytes} [Thu Jul 14 15:37:35 2022] DELETE /identity/v3/users/f1eb0a4d069940cdb2fa6d0c643bacfc => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:37:35.428080 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015590ed-1193-46bd-be53-00f4152ee5e9 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:35.428373 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015590ed-1193-46bd-be53-00f4152ee5e9 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:35.428661 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015590ed-1193-46bd-be53-00f4152ee5e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:35.449111 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-015590ed-1193-46bd-be53-00f4152ee5e9 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:37:35.450634 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-015590ed-1193-46bd-be53-00f4152ee5e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:35.526364 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 983/1977] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:37:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:37:35.536042 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a06e52-3ef8-464b-917b-51bbd5f33dc7 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:35.537731 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-a6a06e52-3ef8-464b-917b-51bbd5f33dc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:35.598033 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a06e52-3ef8-464b-917b-51bbd5f33dc7 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:35.601731 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e19992-2ad7-43f1-8f4d-f7509740cc49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:35.602379 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e19992-2ad7-43f1-8f4d-f7509740cc49 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:35.602712 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e19992-2ad7-43f1-8f4d-f7509740cc49 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:35.602930 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e19992-2ad7-43f1-8f4d-f7509740cc49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:35.613198 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e19992-2ad7-43f1-8f4d-f7509740cc49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:35.614650 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e19992-2ad7-43f1-8f4d-f7509740cc49 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:35.624950 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 995/1978] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:37:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 15:37:35.943441 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015590ed-1193-46bd-be53-00f4152ee5e9 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:35.944910 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-015590ed-1193-46bd-be53-00f4152ee5e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:35.958216 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015590ed-1193-46bd-be53-00f4152ee5e9 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:35.964655 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489b5d22-7fe9-4f32-9fd8-844355d574f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:35.965403 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489b5d22-7fe9-4f32-9fd8-844355d574f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:35.965744 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489b5d22-7fe9-4f32-9fd8-844355d574f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:35.966094 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489b5d22-7fe9-4f32-9fd8-844355d574f2 admin admin] PATH_INFO: `/v3/projects/af7cfa8cb5af4b9a8eaf378d1362dacf` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:35.967428 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489b5d22-7fe9-4f32-9fd8-844355d574f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=af7cfa8cb5af4b9a8eaf378d1362dacf)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:35.970535 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489b5d22-7fe9-4f32-9fd8-844355d574f2 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:36.016863 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:37:36.030544 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-489b5d22-7fe9-4f32-9fd8-844355d574f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af7cfa8cb5af4b9a8eaf378d1362dacf', 'request_id': 'req-489b5d22-7fe9-4f32-9fd8-844355d574f2'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:36.036921 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-489b5d22-7fe9-4f32-9fd8-844355d574f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af7cfa8cb5af4b9a8eaf378d1362dacf', 'request_id': 'req-489b5d22-7fe9-4f32-9fd8-844355d574f2'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:36.042578 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-489b5d22-7fe9-4f32-9fd8-844355d574f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af7cfa8cb5af4b9a8eaf378d1362dacf', 'request_id': 'req-489b5d22-7fe9-4f32-9fd8-844355d574f2'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:36.055782 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-489b5d22-7fe9-4f32-9fd8-844355d574f2 admin admin] The token cache is being invalidate because project af7cfa8cb5af4b9a8eaf378d1362dacf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 15:37:36.056075 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-489b5d22-7fe9-4f32-9fd8-844355d574f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:36.056697 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-489b5d22-7fe9-4f32-9fd8-844355d574f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:36.058273 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 984/1979] 10.222.0.99 () {58 vars in 1232 bytes} [Thu Jul 14 15:37:35 2022] DELETE /identity/v3/projects/af7cfa8cb5af4b9a8eaf378d1362dacf => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:37:36.063168 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e19992-2ad7-43f1-8f4d-f7509740cc49 service neutron] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:36.072923 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e19992-2ad7-43f1-8f4d-f7509740cc49 service neutron] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:36.076547 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:37:36.131296 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff29c4f-11b2-443d-8dd2-a38acf9c379a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:36.132207 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff29c4f-11b2-443d-8dd2-a38acf9c379a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:36.132521 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff29c4f-11b2-443d-8dd2-a38acf9c379a admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:36.132804 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff29c4f-11b2-443d-8dd2-a38acf9c379a admin admin] PATH_INFO: `/v3/users/a7b2d8e27ee84e809b35779311b8caf2` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:36.134012 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff29c4f-11b2-443d-8dd2-a38acf9c379a admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7b2d8e27ee84e809b35779311b8caf2)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:36.136004 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff29c4f-11b2-443d-8dd2-a38acf9c379a admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:36.181702 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-bff29c4f-11b2-443d-8dd2-a38acf9c379a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7b2d8e27ee84e809b35779311b8caf2'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:36.190399 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-bff29c4f-11b2-443d-8dd2-a38acf9c379a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7b2d8e27ee84e809b35779311b8caf2'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:36.206872 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-bff29c4f-11b2-443d-8dd2-a38acf9c379a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7b2d8e27ee84e809b35779311b8caf2'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:36.207578 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-bff29c4f-11b2-443d-8dd2-a38acf9c379a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7b2d8e27ee84e809b35779311b8caf2'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:36.208297 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-bff29c4f-11b2-443d-8dd2-a38acf9c379a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7b2d8e27ee84e809b35779311b8caf2'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:36.221696 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 996/1980] 10.222.0.99 () {58 vars in 1223 bytes} [Thu Jul 14 15:37:36 2022] DELETE /identity/v3/users/a7b2d8e27ee84e809b35779311b8caf2 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:37:36.592187 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489b5d22-7fe9-4f32-9fd8-844355d574f2 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:36.594620 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-489b5d22-7fe9-4f32-9fd8-844355d574f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:36.607865 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489b5d22-7fe9-4f32-9fd8-844355d574f2 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:36.672343 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bb715b-a244-4994-be4b-2d33a5f315e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:36.673218 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bb715b-a244-4994-be4b-2d33a5f315e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:36.673505 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bb715b-a244-4994-be4b-2d33a5f315e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:36.673787 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bb715b-a244-4994-be4b-2d33a5f315e0 admin admin] PATH_INFO: `/v3/projects/7998b6c82ca948faacef764291158e7a` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:36.674986 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bb715b-a244-4994-be4b-2d33a5f315e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7998b6c82ca948faacef764291158e7a)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:36.676944 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bb715b-a244-4994-be4b-2d33a5f315e0 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:36.723908 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-f3bb715b-a244-4994-be4b-2d33a5f315e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7998b6c82ca948faacef764291158e7a', 'request_id': 'req-f3bb715b-a244-4994-be4b-2d33a5f315e0'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:36.729899 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-f3bb715b-a244-4994-be4b-2d33a5f315e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7998b6c82ca948faacef764291158e7a', 'request_id': 'req-f3bb715b-a244-4994-be4b-2d33a5f315e0'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:36.736561 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-f3bb715b-a244-4994-be4b-2d33a5f315e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7998b6c82ca948faacef764291158e7a', 'request_id': 'req-f3bb715b-a244-4994-be4b-2d33a5f315e0'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:36.749708 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-f3bb715b-a244-4994-be4b-2d33a5f315e0 admin admin] The token cache is being invalidate because project 7998b6c82ca948faacef764291158e7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 15:37:36.750000 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-f3bb715b-a244-4994-be4b-2d33a5f315e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:36.750653 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-f3bb715b-a244-4994-be4b-2d33a5f315e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:36.752340 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 985/1981] 10.222.0.99 () {58 vars in 1232 bytes} [Thu Jul 14 15:37:36 2022] DELETE /identity/v3/projects/7998b6c82ca948faacef764291158e7a => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:37:36.757162 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62488ec-27ac-4c0c-9a20-4f48a22552d2 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:36.757675 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62488ec-27ac-4c0c-9a20-4f48a22552d2 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:36.757769 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62488ec-27ac-4c0c-9a20-4f48a22552d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:36.775571 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-d62488ec-27ac-4c0c-9a20-4f48a22552d2 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:37:36.776788 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-d62488ec-27ac-4c0c-9a20-4f48a22552d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:36.835468 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 997/1982] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:37:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 15:37:36.843736 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f98ba84-c111-42f9-bc00-0a8896a1a9e5 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:36.844046 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f98ba84-c111-42f9-bc00-0a8896a1a9e5 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:36.844358 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f98ba84-c111-42f9-bc00-0a8896a1a9e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:36.863215 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-3f98ba84-c111-42f9-bc00-0a8896a1a9e5 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:37:36.864485 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-3f98ba84-c111-42f9-bc00-0a8896a1a9e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:36.872471 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 986/1983] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:37:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 15:37:36.876684 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62488ec-27ac-4c0c-9a20-4f48a22552d2 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:36.886764 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62488ec-27ac-4c0c-9a20-4f48a22552d2 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:36.892326 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14647703-3f67-48be-979d-1e0bf2508f3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:36.893007 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14647703-3f67-48be-979d-1e0bf2508f3a admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:36.893328 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14647703-3f67-48be-979d-1e0bf2508f3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:36.893604 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14647703-3f67-48be-979d-1e0bf2508f3a admin admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:36.894375 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14647703-3f67-48be-979d-1e0bf2508f3a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:37:36.894690 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14647703-3f67-48be-979d-1e0bf2508f3a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:36.896603 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14647703-3f67-48be-979d-1e0bf2508f3a admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:36.909431 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 998/1984] 10.222.0.99 () {58 vars in 1152 bytes} [Thu Jul 14 15:37:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:37:36.913771 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f98ba84-c111-42f9-bc00-0a8896a1a9e5 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:36.924122 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f98ba84-c111-42f9-bc00-0a8896a1a9e5 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:36.928708 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14fdcaf-87b8-46cb-b37b-9580b9d91815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:36.929418 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14fdcaf-87b8-46cb-b37b-9580b9d91815 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:36.929705 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14fdcaf-87b8-46cb-b37b-9580b9d91815 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:36.929980 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14fdcaf-87b8-46cb-b37b-9580b9d91815 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:36.930680 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14fdcaf-87b8-46cb-b37b-9580b9d91815 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:36.932617 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14fdcaf-87b8-46cb-b37b-9580b9d91815 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:36.959471 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 987/1985] 10.222.0.99 () {60 vars in 1152 bytes} [Thu Jul 14 15:37:36 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:37:36.963766 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14647703-3f67-48be-979d-1e0bf2508f3a admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:36.975080 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14647703-3f67-48be-979d-1e0bf2508f3a admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:36.984830 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:37:37.056223 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15ec511-6cc4-4798-b7d8-8312a7be21ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:37.056903 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15ec511-6cc4-4798-b7d8-8312a7be21ae admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:37.057192 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15ec511-6cc4-4798-b7d8-8312a7be21ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:37.057474 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15ec511-6cc4-4798-b7d8-8312a7be21ae admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:37.058309 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15ec511-6cc4-4798-b7d8-8312a7be21ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:37.060348 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15ec511-6cc4-4798-b7d8-8312a7be21ae admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:37.086682 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14fdcaf-87b8-46cb-b37b-9580b9d91815 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:37.087133 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 999/1986] 10.222.0.99 () {60 vars in 1143 bytes} [Thu Jul 14 15:37:36 2022] POST /identity/v3/users => generated 293 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 15:37:37.097570 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14fdcaf-87b8-46cb-b37b-9580b9d91815 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:37.102246 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd46a4d5-d1f9-40bf-8e81-296d251d6096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:37.102937 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd46a4d5-d1f9-40bf-8e81-296d251d6096 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:37.103227 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd46a4d5-d1f9-40bf-8e81-296d251d6096 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:37.103508 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd46a4d5-d1f9-40bf-8e81-296d251d6096 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:37.104276 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd46a4d5-d1f9-40bf-8e81-296d251d6096 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:37.105905 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd46a4d5-d1f9-40bf-8e81-296d251d6096 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:37.117379 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 988/1987] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:37:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:37:37.121402 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15ec511-6cc4-4798-b7d8-8312a7be21ae admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:37.131673 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15ec511-6cc4-4798-b7d8-8312a7be21ae admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:37.136308 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe228d0-0a1f-47ae-a50e-cd3370df8862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:37.136975 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe228d0-0a1f-47ae-a50e-cd3370df8862 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:37.137261 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe228d0-0a1f-47ae-a50e-cd3370df8862 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:37.137542 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe228d0-0a1f-47ae-a50e-cd3370df8862 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:37.138267 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe228d0-0a1f-47ae-a50e-cd3370df8862 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:37.140076 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe228d0-0a1f-47ae-a50e-cd3370df8862 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:37.150548 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1000/1988] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:37:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:37:37.154738 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd46a4d5-d1f9-40bf-8e81-296d251d6096 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:37.165821 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd46a4d5-d1f9-40bf-8e81-296d251d6096 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:37.170440 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230f5669-7801-41f9-b751-81a8bf7e0478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:37.172284 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230f5669-7801-41f9-b751-81a8bf7e0478 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:37.172576 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230f5669-7801-41f9-b751-81a8bf7e0478 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:37.172859 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230f5669-7801-41f9-b751-81a8bf7e0478 admin admin] PATH_INFO: `/v3/projects/5fdda75a56784b97be41c41bcc383793/users/27b188dc9a504cbaa2c928ee1293a3f6/roles/b8c98fddc0fd4436b577fd1d3361bc38` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:37.177448 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:37:37.191254 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230f5669-7801-41f9-b751-81a8bf7e0478 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5fdda75a56784b97be41c41bcc383793, user_id=27b188dc9a504cbaa2c928ee1293a3f6, role_id=b8c98fddc0fd4436b577fd1d3361bc38)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:37.194016 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230f5669-7801-41f9-b751-81a8bf7e0478 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:37.208466 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 989/1989] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 15:37:37 2022] PUT /identity/v3/projects/5fdda75a56784b97be41c41bcc383793/users/27b188dc9a504cbaa2c928ee1293a3f6/roles/b8c98fddc0fd4436b577fd1d3361bc38 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:37:37.219350 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce8bd74-c266-4846-a84d-be19c8d458eb None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:37.219673 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce8bd74-c266-4846-a84d-be19c8d458eb None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:37.219957 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce8bd74-c266-4846-a84d-be19c8d458eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:37.246063 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-bce8bd74-c266-4846-a84d-be19c8d458eb None None] MFA Rules not processed for user `27b188dc9a504cbaa2c928ee1293a3f6`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:37:37.247189 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-bce8bd74-c266-4846-a84d-be19c8d458eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:37.314242 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1001/1990] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:37:37 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:37:37.319067 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230f5669-7801-41f9-b751-81a8bf7e0478 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:37.382538 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230f5669-7801-41f9-b751-81a8bf7e0478 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:37.387273 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac948c8-eb68-4661-96b6-7f22204875aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:37.387968 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac948c8-eb68-4661-96b6-7f22204875aa admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:37.388256 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac948c8-eb68-4661-96b6-7f22204875aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:37.388537 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac948c8-eb68-4661-96b6-7f22204875aa admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:37.389748 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac948c8-eb68-4661-96b6-7f22204875aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:37.391791 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac948c8-eb68-4661-96b6-7f22204875aa admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:37.421915 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 990/1991] 10.222.0.99 () {60 vars in 1152 bytes} [Thu Jul 14 15:37:37 2022] POST /identity/v3/projects => generated 387 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:37:37.423067 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce8bd74-c266-4846-a84d-be19c8d458eb None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:37.433541 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce8bd74-c266-4846-a84d-be19c8d458eb None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:37.491708 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6325ff-a1f9-4ab5-a21f-44eef692776a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:37.492432 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6325ff-a1f9-4ab5-a21f-44eef692776a admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:37.492721 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6325ff-a1f9-4ab5-a21f-44eef692776a admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:37.493007 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6325ff-a1f9-4ab5-a21f-44eef692776a admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:37.493820 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6325ff-a1f9-4ab5-a21f-44eef692776a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:37.495848 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6325ff-a1f9-4ab5-a21f-44eef692776a admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:37.517167 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1002/1992] 10.222.0.99 () {60 vars in 1143 bytes} [Thu Jul 14 15:37:37 2022] POST /identity/v3/users => generated 294 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:37:37.521218 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac948c8-eb68-4661-96b6-7f22204875aa admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:37.532405 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac948c8-eb68-4661-96b6-7f22204875aa admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:37.537019 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df35f90f-5cf8-48da-8548-3c97eab381d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:37.537688 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df35f90f-5cf8-48da-8548-3c97eab381d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:37.537977 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df35f90f-5cf8-48da-8548-3c97eab381d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:37.538260 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df35f90f-5cf8-48da-8548-3c97eab381d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:37.538990 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df35f90f-5cf8-48da-8548-3c97eab381d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:37.540636 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df35f90f-5cf8-48da-8548-3c97eab381d1 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:37.552650 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 991/1993] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:37:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:37:37.556860 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6325ff-a1f9-4ab5-a21f-44eef692776a admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:37.567773 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6325ff-a1f9-4ab5-a21f-44eef692776a admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:37.572459 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ab4ad5-c77d-4817-81b2-122e0ff89f02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:37.573131 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ab4ad5-c77d-4817-81b2-122e0ff89f02 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:37.573508 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ab4ad5-c77d-4817-81b2-122e0ff89f02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:37.573703 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ab4ad5-c77d-4817-81b2-122e0ff89f02 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:37.574463 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ab4ad5-c77d-4817-81b2-122e0ff89f02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:37.576110 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ab4ad5-c77d-4817-81b2-122e0ff89f02 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:37.582183 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:37:37.587721 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1003/1994] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:37:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 15:37:37.592178 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df35f90f-5cf8-48da-8548-3c97eab381d1 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:37.603629 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df35f90f-5cf8-48da-8548-3c97eab381d1 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:37.608500 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f50a170-4e35-4fe2-8f3e-c5ed01fcad66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:37.609553 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f50a170-4e35-4fe2-8f3e-c5ed01fcad66 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:37.609812 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f50a170-4e35-4fe2-8f3e-c5ed01fcad66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:37.610439 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f50a170-4e35-4fe2-8f3e-c5ed01fcad66 admin admin] PATH_INFO: `/v3/projects/20f66798acc247a280d0cd3a6619c676/users/8246ba65dd934e9fb75670bb4bb5cadd/roles/b8c98fddc0fd4436b577fd1d3361bc38` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:37.626387 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f50a170-4e35-4fe2-8f3e-c5ed01fcad66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20f66798acc247a280d0cd3a6619c676, user_id=8246ba65dd934e9fb75670bb4bb5cadd, role_id=b8c98fddc0fd4436b577fd1d3361bc38)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:37.629159 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f50a170-4e35-4fe2-8f3e-c5ed01fcad66 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:37.644422 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 992/1995] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 15:37:37 2022] PUT /identity/v3/projects/20f66798acc247a280d0cd3a6619c676/users/8246ba65dd934e9fb75670bb4bb5cadd/roles/b8c98fddc0fd4436b577fd1d3361bc38 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 15:37:37.656860 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58673e12-c725-4d4f-8996-df26556af89a None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:37.657158 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58673e12-c725-4d4f-8996-df26556af89a None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:37.657441 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58673e12-c725-4d4f-8996-df26556af89a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:37.685060 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-58673e12-c725-4d4f-8996-df26556af89a None None] MFA Rules not processed for user `8246ba65dd934e9fb75670bb4bb5cadd`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:37:37.686309 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-58673e12-c725-4d4f-8996-df26556af89a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:37.764033 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1004/1996] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:37:37 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:37:37.769646 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f50a170-4e35-4fe2-8f3e-c5ed01fcad66 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:37.831513 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f50a170-4e35-4fe2-8f3e-c5ed01fcad66 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:37.836326 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df93b53b-67be-4af9-ba58-ac2ec50e945f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:37.836989 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df93b53b-67be-4af9-ba58-ac2ec50e945f admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:37.837281 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df93b53b-67be-4af9-ba58-ac2ec50e945f admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:37.837564 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df93b53b-67be-4af9-ba58-ac2ec50e945f admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:37.838290 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df93b53b-67be-4af9-ba58-ac2ec50e945f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:37.840247 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df93b53b-67be-4af9-ba58-ac2ec50e945f admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:37.867302 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 993/1997] 10.222.0.99 () {60 vars in 1152 bytes} [Thu Jul 14 15:37:37 2022] POST /identity/v3/projects => generated 387 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 15:37:37.870764 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58673e12-c725-4d4f-8996-df26556af89a None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:37.881371 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58673e12-c725-4d4f-8996-df26556af89a None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:37.941831 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd30f44-5243-40b9-b0e9-26f51f0013e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:37.942500 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd30f44-5243-40b9-b0e9-26f51f0013e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:37.942792 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd30f44-5243-40b9-b0e9-26f51f0013e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:37.943071 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd30f44-5243-40b9-b0e9-26f51f0013e4 admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:37.943799 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd30f44-5243-40b9-b0e9-26f51f0013e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:37.945712 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd30f44-5243-40b9-b0e9-26f51f0013e4 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:37.966586 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1005/1998] 10.222.0.99 () {60 vars in 1143 bytes} [Thu Jul 14 15:37:37 2022] POST /identity/v3/users => generated 300 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:37:37.971444 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df93b53b-67be-4af9-ba58-ac2ec50e945f admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:37.981773 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df93b53b-67be-4af9-ba58-ac2ec50e945f admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:37.986432 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b48a62a-0205-4b94-a219-5a247efda07d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:37.987096 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b48a62a-0205-4b94-a219-5a247efda07d admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:37.987388 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b48a62a-0205-4b94-a219-5a247efda07d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:37.987710 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b48a62a-0205-4b94-a219-5a247efda07d admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:37.988434 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b48a62a-0205-4b94-a219-5a247efda07d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:37.990061 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b48a62a-0205-4b94-a219-5a247efda07d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:38.001195 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 994/1999] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:37:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:37:38.005475 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd30f44-5243-40b9-b0e9-26f51f0013e4 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:38.016418 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd30f44-5243-40b9-b0e9-26f51f0013e4 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:38.021303 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48765547-2695-4aa3-ad13-feccb6c03d74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:38.022184 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48765547-2695-4aa3-ad13-feccb6c03d74 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:38.022470 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48765547-2695-4aa3-ad13-feccb6c03d74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:38.022752 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48765547-2695-4aa3-ad13-feccb6c03d74 admin admin] PATH_INFO: `/v3/projects/a2b6c40ad0fa437291314051caac234c/users/e30c81d425014cfe821aad836294816c/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:38.038367 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48765547-2695-4aa3-ad13-feccb6c03d74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2b6c40ad0fa437291314051caac234c, user_id=e30c81d425014cfe821aad836294816c, role_id=d0f6ba1314f14d9f9c816bdb5d0c3aac)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:38.041061 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48765547-2695-4aa3-ad13-feccb6c03d74 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:38.054709 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1006/2000] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 15:37:38 2022] PUT /identity/v3/projects/a2b6c40ad0fa437291314051caac234c/users/e30c81d425014cfe821aad836294816c/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 15:37:38.058809 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b48a62a-0205-4b94-a219-5a247efda07d admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:38.070506 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b48a62a-0205-4b94-a219-5a247efda07d admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:38.075516 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:37:38.133267 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb558ba-764c-43a3-845e-e75a9850e4f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:38.133943 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb558ba-764c-43a3-845e-e75a9850e4f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:38.134230 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb558ba-764c-43a3-845e-e75a9850e4f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:38.134511 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb558ba-764c-43a3-845e-e75a9850e4f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:38.135251 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb558ba-764c-43a3-845e-e75a9850e4f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:38.136904 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb558ba-764c-43a3-845e-e75a9850e4f9 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:38.147942 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 995/2001] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:37:38 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:37:38.152062 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48765547-2695-4aa3-ad13-feccb6c03d74 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:38.162897 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48765547-2695-4aa3-ad13-feccb6c03d74 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:38.167416 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73501860-d4ca-4c45-9137-63746710a143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:38.168334 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73501860-d4ca-4c45-9137-63746710a143 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:38.168619 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73501860-d4ca-4c45-9137-63746710a143 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:38.168896 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73501860-d4ca-4c45-9137-63746710a143 admin admin] PATH_INFO: `/v3/projects/a2b6c40ad0fa437291314051caac234c/users/e30c81d425014cfe821aad836294816c/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:38.170964 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73501860-d4ca-4c45-9137-63746710a143 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2b6c40ad0fa437291314051caac234c, user_id=e30c81d425014cfe821aad836294816c, role_id=d0f6ba1314f14d9f9c816bdb5d0c3aac)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:38.173811 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73501860-d4ca-4c45-9137-63746710a143 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:38.184514 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1007/2002] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 15:37:38 2022] PUT /identity/v3/projects/a2b6c40ad0fa437291314051caac234c/users/e30c81d425014cfe821aad836294816c/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:37:38.194743 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f55180-8c01-4460-bdeb-e9e0b9466bee None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:38.195037 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f55180-8c01-4460-bdeb-e9e0b9466bee None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:38.195321 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f55180-8c01-4460-bdeb-e9e0b9466bee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:38.223787 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-e7f55180-8c01-4460-bdeb-e9e0b9466bee None None] MFA Rules not processed for user `e30c81d425014cfe821aad836294816c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:37:38.225029 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e7f55180-8c01-4460-bdeb-e9e0b9466bee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:38.298922 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 996/2003] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:37:38 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:37:38.309771 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73501860-d4ca-4c45-9137-63746710a143 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:38.311625 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-73501860-d4ca-4c45-9137-63746710a143 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:38.359923 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:37:38.374486 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73501860-d4ca-4c45-9137-63746710a143 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:38.378375 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4787be63-86f7-4136-ae47-d5ebda6b6b44 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3e1a2b83f3d4d88ba5084dfaffd22f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:38.379027 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4787be63-86f7-4136-ae47-d5ebda6b6b44 service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:38.379322 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4787be63-86f7-4136-ae47-d5ebda6b6b44 service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:38.379620 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4787be63-86f7-4136-ae47-d5ebda6b6b44 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:38.388758 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4787be63-86f7-4136-ae47-d5ebda6b6b44 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:38.390976 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4787be63-86f7-4136-ae47-d5ebda6b6b44 service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:38.401511 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1008/2004] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:37:38 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:37:38.709262 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f55180-8c01-4460-bdeb-e9e0b9466bee None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:38.730214 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f55180-8c01-4460-bdeb-e9e0b9466bee None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:38.737515 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561c0f30-14c5-429a-a84e-f552b749b589 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3e1a2b83f3d4d88ba5084dfaffd22f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:38.737710 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561c0f30-14c5-429a-a84e-f552b749b589 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:38.738119 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561c0f30-14c5-429a-a84e-f552b749b589 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:38.738413 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561c0f30-14c5-429a-a84e-f552b749b589 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:38.740680 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-561c0f30-14c5-429a-a84e-f552b749b589 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:38.866949 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561c0f30-14c5-429a-a84e-f552b749b589 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:38.869435 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561c0f30-14c5-429a-a84e-f552b749b589 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:38.881867 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 997/2005] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:37:38 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:37:39.030494 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4787be63-86f7-4136-ae47-d5ebda6b6b44 service cinder] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:39.032053 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-4787be63-86f7-4136-ae47-d5ebda6b6b44 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:39.043356 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4787be63-86f7-4136-ae47-d5ebda6b6b44 service cinder] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:39.049682 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd66f481-071d-47bb-b736-5f218d56ee26 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3e1a2b83f3d4d88ba5084dfaffd22f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:39.050352 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd66f481-071d-47bb-b736-5f218d56ee26 service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:39.050642 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd66f481-071d-47bb-b736-5f218d56ee26 service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:39.050919 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd66f481-071d-47bb-b736-5f218d56ee26 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:39.052725 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-bd66f481-071d-47bb-b736-5f218d56ee26 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:39.143908 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd66f481-071d-47bb-b736-5f218d56ee26 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:39.145389 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd66f481-071d-47bb-b736-5f218d56ee26 service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:39.155996 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1009/2006] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:37:39 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:37:39.291794 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561c0f30-14c5-429a-a84e-f552b749b589 service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:39.301615 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561c0f30-14c5-429a-a84e-f552b749b589 service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:39.305492 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d0cc7a-1d17-47d3-9815-5401717188a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3e1a2b83f3d4d88ba5084dfaffd22f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:39.306148 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d0cc7a-1d17-47d3-9815-5401717188a3 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:39.306464 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d0cc7a-1d17-47d3-9815-5401717188a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:39.306751 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d0cc7a-1d17-47d3-9815-5401717188a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:39.331188 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:37:39.372325 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d0cc7a-1d17-47d3-9815-5401717188a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:39.374572 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d0cc7a-1d17-47d3-9815-5401717188a3 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:39.385356 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 998/2007] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:37:39 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:37:49.151841 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=37 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:37:49.380278 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=17 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:37:52.099420 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd66f481-071d-47bb-b736-5f218d56ee26 service cinder] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:52.108013 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd66f481-071d-47bb-b736-5f218d56ee26 service cinder] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:52.111778 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b84f65-3208-4c88-a1e6-a4835576203e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3e1a2b83f3d4d88ba5084dfaffd22f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:52.112430 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b84f65-3208-4c88-a1e6-a4835576203e service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:52.112737 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b84f65-3208-4c88-a1e6-a4835576203e service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:52.112985 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b84f65-3208-4c88-a1e6-a4835576203e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:52.182165 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b84f65-3208-4c88-a1e6-a4835576203e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:52.183818 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b84f65-3208-4c88-a1e6-a4835576203e service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:52.193962 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1010/2008] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:37:52 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:37:54.568504 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d0cc7a-1d17-47d3-9815-5401717188a3 service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:54.640914 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d0cc7a-1d17-47d3-9815-5401717188a3 service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:54.645491 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f082a1ff-bbb7-4b84-b8f7-da3b4833a29e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:54.646628 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f082a1ff-bbb7-4b84-b8f7-da3b4833a29e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:54.646919 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f082a1ff-bbb7-4b84-b8f7-da3b4833a29e admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:54.647199 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f082a1ff-bbb7-4b84-b8f7-da3b4833a29e admin admin] PATH_INFO: `/v3/users/27b188dc9a504cbaa2c928ee1293a3f6` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:54.648438 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f082a1ff-bbb7-4b84-b8f7-da3b4833a29e admin admin] RBAC: Authorizing `identity:delete_user(user_id=27b188dc9a504cbaa2c928ee1293a3f6)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:54.650374 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f082a1ff-bbb7-4b84-b8f7-da3b4833a29e admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:54.697615 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-f082a1ff-bbb7-4b84-b8f7-da3b4833a29e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27b188dc9a504cbaa2c928ee1293a3f6'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:54.705162 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-f082a1ff-bbb7-4b84-b8f7-da3b4833a29e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27b188dc9a504cbaa2c928ee1293a3f6'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:54.717057 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-f082a1ff-bbb7-4b84-b8f7-da3b4833a29e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27b188dc9a504cbaa2c928ee1293a3f6'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:54.718214 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-f082a1ff-bbb7-4b84-b8f7-da3b4833a29e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27b188dc9a504cbaa2c928ee1293a3f6'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:54.718838 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-f082a1ff-bbb7-4b84-b8f7-da3b4833a29e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27b188dc9a504cbaa2c928ee1293a3f6'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:54.731240 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 999/2009] 10.222.0.99 () {58 vars in 1223 bytes} [Thu Jul 14 15:37:54 2022] DELETE /identity/v3/users/27b188dc9a504cbaa2c928ee1293a3f6 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:37:54.736714 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b8f35a-9073-4bb2-8328-10578582c398 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:54.737217 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b8f35a-9073-4bb2-8328-10578582c398 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:54.737321 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b8f35a-9073-4bb2-8328-10578582c398 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:54.754706 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-52b8f35a-9073-4bb2-8328-10578582c398 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:37:54.756850 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-52b8f35a-9073-4bb2-8328-10578582c398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:54.822736 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1011/2010] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:37:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 15:37:54.833114 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f082a1ff-bbb7-4b84-b8f7-da3b4833a29e admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:54.834522 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f082a1ff-bbb7-4b84-b8f7-da3b4833a29e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:54.892026 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f082a1ff-bbb7-4b84-b8f7-da3b4833a29e admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:54.895638 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ef6c42-9f8d-4f99-b7bb-4dc0e4a26e4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:54.896536 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ef6c42-9f8d-4f99-b7bb-4dc0e4a26e4c service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:54.896820 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ef6c42-9f8d-4f99-b7bb-4dc0e4a26e4c service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:54.897097 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ef6c42-9f8d-4f99-b7bb-4dc0e4a26e4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:54.907206 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ef6c42-9f8d-4f99-b7bb-4dc0e4a26e4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:54.908697 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ef6c42-9f8d-4f99-b7bb-4dc0e4a26e4c service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:54.918547 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1000/2011] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:37:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:37:55.195696 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b8f35a-9073-4bb2-8328-10578582c398 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:55.197237 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-52b8f35a-9073-4bb2-8328-10578582c398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:55.210632 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b8f35a-9073-4bb2-8328-10578582c398 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:55.215237 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871fcab3-1d0a-4f1d-80ff-ed8b60f81ee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:55.216154 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871fcab3-1d0a-4f1d-80ff-ed8b60f81ee0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:55.216751 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871fcab3-1d0a-4f1d-80ff-ed8b60f81ee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:55.216751 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871fcab3-1d0a-4f1d-80ff-ed8b60f81ee0 admin admin] PATH_INFO: `/v3/projects/5fdda75a56784b97be41c41bcc383793` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:55.218284 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871fcab3-1d0a-4f1d-80ff-ed8b60f81ee0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5fdda75a56784b97be41c41bcc383793)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:55.219973 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871fcab3-1d0a-4f1d-80ff-ed8b60f81ee0 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:55.289637 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-871fcab3-1d0a-4f1d-80ff-ed8b60f81ee0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5fdda75a56784b97be41c41bcc383793', 'request_id': 'req-871fcab3-1d0a-4f1d-80ff-ed8b60f81ee0'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:55.302276 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-871fcab3-1d0a-4f1d-80ff-ed8b60f81ee0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5fdda75a56784b97be41c41bcc383793', 'request_id': 'req-871fcab3-1d0a-4f1d-80ff-ed8b60f81ee0'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:55.308277 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-871fcab3-1d0a-4f1d-80ff-ed8b60f81ee0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5fdda75a56784b97be41c41bcc383793', 'request_id': 'req-871fcab3-1d0a-4f1d-80ff-ed8b60f81ee0'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:55.325320 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-871fcab3-1d0a-4f1d-80ff-ed8b60f81ee0 admin admin] The token cache is being invalidate because project 5fdda75a56784b97be41c41bcc383793 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 15:37:55.325621 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-871fcab3-1d0a-4f1d-80ff-ed8b60f81ee0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:55.326284 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-871fcab3-1d0a-4f1d-80ff-ed8b60f81ee0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:55.328939 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1012/2012] 10.222.0.99 () {58 vars in 1232 bytes} [Thu Jul 14 15:37:55 2022] DELETE /identity/v3/projects/5fdda75a56784b97be41c41bcc383793 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:37:55.332998 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ef6c42-9f8d-4f99-b7bb-4dc0e4a26e4c service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:55.345583 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ef6c42-9f8d-4f99-b7bb-4dc0e4a26e4c service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:55.372587 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:37:55.418066 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f443688e-43f7-47e2-96cd-252ebdea4e4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:55.418961 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f443688e-43f7-47e2-96cd-252ebdea4e4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:55.419341 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f443688e-43f7-47e2-96cd-252ebdea4e4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:55.419525 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f443688e-43f7-47e2-96cd-252ebdea4e4f admin admin] PATH_INFO: `/v3/users/8246ba65dd934e9fb75670bb4bb5cadd` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:55.420822 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f443688e-43f7-47e2-96cd-252ebdea4e4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=8246ba65dd934e9fb75670bb4bb5cadd)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:55.422801 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f443688e-43f7-47e2-96cd-252ebdea4e4f admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:55.477228 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-f443688e-43f7-47e2-96cd-252ebdea4e4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8246ba65dd934e9fb75670bb4bb5cadd'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:55.485718 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-f443688e-43f7-47e2-96cd-252ebdea4e4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8246ba65dd934e9fb75670bb4bb5cadd'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:55.498876 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-f443688e-43f7-47e2-96cd-252ebdea4e4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8246ba65dd934e9fb75670bb4bb5cadd'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:55.500359 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-f443688e-43f7-47e2-96cd-252ebdea4e4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8246ba65dd934e9fb75670bb4bb5cadd'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:55.500989 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-f443688e-43f7-47e2-96cd-252ebdea4e4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8246ba65dd934e9fb75670bb4bb5cadd'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:55.513276 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1001/2013] 10.222.0.99 () {58 vars in 1223 bytes} [Thu Jul 14 15:37:55 2022] DELETE /identity/v3/users/8246ba65dd934e9fb75670bb4bb5cadd => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:37:55.791713 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871fcab3-1d0a-4f1d-80ff-ed8b60f81ee0 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:55.793222 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-871fcab3-1d0a-4f1d-80ff-ed8b60f81ee0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:55.809003 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871fcab3-1d0a-4f1d-80ff-ed8b60f81ee0 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:55.832832 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:37:55.879779 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702521a2-b541-426a-8558-bc305bff8e1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:55.880657 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702521a2-b541-426a-8558-bc305bff8e1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:55.880938 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702521a2-b541-426a-8558-bc305bff8e1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:55.881233 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702521a2-b541-426a-8558-bc305bff8e1f admin admin] PATH_INFO: `/v3/projects/20f66798acc247a280d0cd3a6619c676` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:55.882462 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702521a2-b541-426a-8558-bc305bff8e1f admin admin] RBAC: Authorizing `identity:delete_project(project_id=20f66798acc247a280d0cd3a6619c676)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:55.884424 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702521a2-b541-426a-8558-bc305bff8e1f admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:55.938527 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-702521a2-b541-426a-8558-bc305bff8e1f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20f66798acc247a280d0cd3a6619c676', 'request_id': 'req-702521a2-b541-426a-8558-bc305bff8e1f'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:55.944686 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-702521a2-b541-426a-8558-bc305bff8e1f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20f66798acc247a280d0cd3a6619c676', 'request_id': 'req-702521a2-b541-426a-8558-bc305bff8e1f'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:55.950163 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-702521a2-b541-426a-8558-bc305bff8e1f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20f66798acc247a280d0cd3a6619c676', 'request_id': 'req-702521a2-b541-426a-8558-bc305bff8e1f'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:55.965251 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-702521a2-b541-426a-8558-bc305bff8e1f admin admin] The token cache is being invalidate because project 20f66798acc247a280d0cd3a6619c676 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 15:37:55.965538 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-702521a2-b541-426a-8558-bc305bff8e1f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:55.966177 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-702521a2-b541-426a-8558-bc305bff8e1f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:55.968041 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1013/2014] 10.222.0.99 () {58 vars in 1232 bytes} [Thu Jul 14 15:37:55 2022] DELETE /identity/v3/projects/20f66798acc247a280d0cd3a6619c676 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 15:37:55.972289 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f443688e-43f7-47e2-96cd-252ebdea4e4f admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:55.973737 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f443688e-43f7-47e2-96cd-252ebdea4e4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:55.987733 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f443688e-43f7-47e2-96cd-252ebdea4e4f admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:56.051168 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d65b80-0706-451e-b386-f5d053eea213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:56.052116 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d65b80-0706-451e-b386-f5d053eea213 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:56.052405 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d65b80-0706-451e-b386-f5d053eea213 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:56.052683 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d65b80-0706-451e-b386-f5d053eea213 admin admin] PATH_INFO: `/v3/users/e30c81d425014cfe821aad836294816c` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:56.053883 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d65b80-0706-451e-b386-f5d053eea213 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e30c81d425014cfe821aad836294816c)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:56.055857 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d65b80-0706-451e-b386-f5d053eea213 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:56.101280 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:37:56.103952 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-74d65b80-0706-451e-b386-f5d053eea213 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e30c81d425014cfe821aad836294816c'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:56.112370 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-74d65b80-0706-451e-b386-f5d053eea213 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e30c81d425014cfe821aad836294816c'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:56.124255 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-74d65b80-0706-451e-b386-f5d053eea213 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e30c81d425014cfe821aad836294816c'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:56.125952 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-74d65b80-0706-451e-b386-f5d053eea213 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e30c81d425014cfe821aad836294816c'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:56.126776 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-74d65b80-0706-451e-b386-f5d053eea213 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e30c81d425014cfe821aad836294816c'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:56.139529 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1002/2015] 10.222.0.99 () {58 vars in 1223 bytes} [Thu Jul 14 15:37:55 2022] DELETE /identity/v3/users/e30c81d425014cfe821aad836294816c => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 15:37:56.449752 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702521a2-b541-426a-8558-bc305bff8e1f admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:56.453184 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-702521a2-b541-426a-8558-bc305bff8e1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:56.473545 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702521a2-b541-426a-8558-bc305bff8e1f admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:56.537224 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c71686f-2d5e-4936-9d31-b2993c838858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:56.538198 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c71686f-2d5e-4936-9d31-b2993c838858 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:56.538554 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c71686f-2d5e-4936-9d31-b2993c838858 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:56.538905 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c71686f-2d5e-4936-9d31-b2993c838858 admin admin] PATH_INFO: `/v3/projects/a2b6c40ad0fa437291314051caac234c` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:56.540151 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c71686f-2d5e-4936-9d31-b2993c838858 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2b6c40ad0fa437291314051caac234c)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:56.542223 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c71686f-2d5e-4936-9d31-b2993c838858 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:56.592982 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-7c71686f-2d5e-4936-9d31-b2993c838858 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2b6c40ad0fa437291314051caac234c', 'request_id': 'req-7c71686f-2d5e-4936-9d31-b2993c838858'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:56.599718 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-7c71686f-2d5e-4936-9d31-b2993c838858 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2b6c40ad0fa437291314051caac234c', 'request_id': 'req-7c71686f-2d5e-4936-9d31-b2993c838858'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:56.601105 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:37:56.606667 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-7c71686f-2d5e-4936-9d31-b2993c838858 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2b6c40ad0fa437291314051caac234c', 'request_id': 'req-7c71686f-2d5e-4936-9d31-b2993c838858'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:56.621118 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-7c71686f-2d5e-4936-9d31-b2993c838858 admin admin] The token cache is being invalidate because project a2b6c40ad0fa437291314051caac234c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 15:37:56.621414 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-7c71686f-2d5e-4936-9d31-b2993c838858 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:56.622123 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-7c71686f-2d5e-4936-9d31-b2993c838858 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:56.623775 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1014/2016] 10.222.0.99 () {58 vars in 1232 bytes} [Thu Jul 14 15:37:56 2022] DELETE /identity/v3/projects/a2b6c40ad0fa437291314051caac234c => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:37:56.632172 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddd84da-00a0-41ab-9dcb-af9c188548bb None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:56.632473 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddd84da-00a0-41ab-9dcb-af9c188548bb None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:56.632770 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddd84da-00a0-41ab-9dcb-af9c188548bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:56.653182 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-5ddd84da-00a0-41ab-9dcb-af9c188548bb None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:37:56.654416 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-5ddd84da-00a0-41ab-9dcb-af9c188548bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:56.730538 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1003/2017] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:37:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:37:56.743859 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015393e4-5b3e-4cb3-836c-ed997fb16a36 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:56.744159 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015393e4-5b3e-4cb3-836c-ed997fb16a36 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:56.744447 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015393e4-5b3e-4cb3-836c-ed997fb16a36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:56.767828 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-015393e4-5b3e-4cb3-836c-ed997fb16a36 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:37:56.769410 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-015393e4-5b3e-4cb3-836c-ed997fb16a36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:56.780392 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1015/2018] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:37:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:37:56.785459 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddd84da-00a0-41ab-9dcb-af9c188548bb None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:56.796142 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddd84da-00a0-41ab-9dcb-af9c188548bb None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:56.801840 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41208d2-847f-472f-a48f-5ef1cd84e70b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:56.802536 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41208d2-847f-472f-a48f-5ef1cd84e70b admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:56.802829 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41208d2-847f-472f-a48f-5ef1cd84e70b admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:56.803113 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41208d2-847f-472f-a48f-5ef1cd84e70b admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:56.803905 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41208d2-847f-472f-a48f-5ef1cd84e70b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:37:56.804226 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41208d2-847f-472f-a48f-5ef1cd84e70b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:56.805881 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41208d2-847f-472f-a48f-5ef1cd84e70b admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:56.822911 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1004/2019] 10.222.0.99 () {58 vars in 1152 bytes} [Thu Jul 14 15:37:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:37:56.824272 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015393e4-5b3e-4cb3-836c-ed997fb16a36 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:56.835315 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015393e4-5b3e-4cb3-836c-ed997fb16a36 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:56.840030 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4905a7e1-a126-4b99-81da-2c9c8f3ad98e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:56.840715 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4905a7e1-a126-4b99-81da-2c9c8f3ad98e admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:56.841007 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4905a7e1-a126-4b99-81da-2c9c8f3ad98e admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:56.841291 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4905a7e1-a126-4b99-81da-2c9c8f3ad98e admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:56.842002 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4905a7e1-a126-4b99-81da-2c9c8f3ad98e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:56.845291 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4905a7e1-a126-4b99-81da-2c9c8f3ad98e admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:56.875239 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1016/2020] 10.222.0.99 () {60 vars in 1152 bytes} [Thu Jul 14 15:37:56 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:37:56.878978 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41208d2-847f-472f-a48f-5ef1cd84e70b admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:56.889529 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41208d2-847f-472f-a48f-5ef1cd84e70b admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:56.962499 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f50fe76-f3b2-4a53-8372-d386b1998bbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:56.963364 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f50fe76-f3b2-4a53-8372-d386b1998bbd admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:56.963689 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f50fe76-f3b2-4a53-8372-d386b1998bbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:56.963977 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f50fe76-f3b2-4a53-8372-d386b1998bbd admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:56.964709 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f50fe76-f3b2-4a53-8372-d386b1998bbd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:56.966651 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f50fe76-f3b2-4a53-8372-d386b1998bbd admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:56.993051 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1005/2021] 10.222.0.99 () {60 vars in 1143 bytes} [Thu Jul 14 15:37:56 2022] POST /identity/v3/users => generated 292 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 15:37:56.994248 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4905a7e1-a126-4b99-81da-2c9c8f3ad98e admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:57.004959 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4905a7e1-a126-4b99-81da-2c9c8f3ad98e admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:57.009662 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82e66cd-2673-4618-8630-b3bba02de3b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:57.010329 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82e66cd-2673-4618-8630-b3bba02de3b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:57.010613 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82e66cd-2673-4618-8630-b3bba02de3b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:57.010892 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82e66cd-2673-4618-8630-b3bba02de3b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:57.011692 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82e66cd-2673-4618-8630-b3bba02de3b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:57.013296 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82e66cd-2673-4618-8630-b3bba02de3b7 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:57.024863 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1017/2022] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:37:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 15:37:57.028788 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f50fe76-f3b2-4a53-8372-d386b1998bbd admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:57.039726 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f50fe76-f3b2-4a53-8372-d386b1998bbd admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:57.044428 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e05df97-728d-4371-b106-f222393dc564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:57.045123 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e05df97-728d-4371-b106-f222393dc564 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:57.045415 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e05df97-728d-4371-b106-f222393dc564 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:57.045701 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e05df97-728d-4371-b106-f222393dc564 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:57.046442 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e05df97-728d-4371-b106-f222393dc564 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:57.048937 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e05df97-728d-4371-b106-f222393dc564 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:57.059145 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1006/2023] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:37:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:37:57.063494 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82e66cd-2673-4618-8630-b3bba02de3b7 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:57.073818 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82e66cd-2673-4618-8630-b3bba02de3b7 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:57.079287 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8f6f37-aa9b-4801-bbe5-29073e17e8e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:57.080539 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8f6f37-aa9b-4801-bbe5-29073e17e8e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:57.080833 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8f6f37-aa9b-4801-bbe5-29073e17e8e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:57.081117 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8f6f37-aa9b-4801-bbe5-29073e17e8e5 admin admin] PATH_INFO: `/v3/projects/6f3ed484beff472d899c8e13f263f426/users/281a4bcae25349f89bf92234410c45a7/roles/b8c98fddc0fd4436b577fd1d3361bc38` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:57.098375 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8f6f37-aa9b-4801-bbe5-29073e17e8e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f3ed484beff472d899c8e13f263f426, user_id=281a4bcae25349f89bf92234410c45a7, role_id=b8c98fddc0fd4436b577fd1d3361bc38)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:57.101165 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8f6f37-aa9b-4801-bbe5-29073e17e8e5 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:57.114531 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1018/2024] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 15:37:57 2022] PUT /identity/v3/projects/6f3ed484beff472d899c8e13f263f426/users/281a4bcae25349f89bf92234410c45a7/roles/b8c98fddc0fd4436b577fd1d3361bc38 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:37:57.125951 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609f5943-0956-46d1-aa6b-3061689b3e79 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:57.126262 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609f5943-0956-46d1-aa6b-3061689b3e79 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:57.126547 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609f5943-0956-46d1-aa6b-3061689b3e79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:57.154832 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-609f5943-0956-46d1-aa6b-3061689b3e79 None None] MFA Rules not processed for user `281a4bcae25349f89bf92234410c45a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:37:57.156909 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-609f5943-0956-46d1-aa6b-3061689b3e79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:57.169459 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:37:57.224829 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1007/2025] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:37:57 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:37:57.229365 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8f6f37-aa9b-4801-bbe5-29073e17e8e5 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:57.290654 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8f6f37-aa9b-4801-bbe5-29073e17e8e5 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:57.295378 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce04a682-fe63-4c47-9930-740dbe311d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:57.296098 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce04a682-fe63-4c47-9930-740dbe311d16 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:57.296393 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce04a682-fe63-4c47-9930-740dbe311d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:57.296812 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce04a682-fe63-4c47-9930-740dbe311d16 admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:57.297985 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce04a682-fe63-4c47-9930-740dbe311d16 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:57.299390 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce04a682-fe63-4c47-9930-740dbe311d16 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:57.324684 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1019/2026] 10.222.0.99 () {60 vars in 1152 bytes} [Thu Jul 14 15:37:57 2022] POST /identity/v3/projects => generated 381 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:37:57.329462 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609f5943-0956-46d1-aa6b-3061689b3e79 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:57.341285 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609f5943-0956-46d1-aa6b-3061689b3e79 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:57.400992 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91109f12-0ed5-4f89-9641-4d8be8a0c80b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:57.401671 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91109f12-0ed5-4f89-9641-4d8be8a0c80b admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:57.401960 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91109f12-0ed5-4f89-9641-4d8be8a0c80b admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:57.402239 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91109f12-0ed5-4f89-9641-4d8be8a0c80b admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:57.402946 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91109f12-0ed5-4f89-9641-4d8be8a0c80b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:57.405010 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91109f12-0ed5-4f89-9641-4d8be8a0c80b admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:57.431161 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1008/2027] 10.222.0.99 () {60 vars in 1143 bytes} [Thu Jul 14 15:37:57 2022] POST /identity/v3/users => generated 297 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 15:37:57.432424 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce04a682-fe63-4c47-9930-740dbe311d16 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:57.444214 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce04a682-fe63-4c47-9930-740dbe311d16 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:57.448931 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc88d378-cd38-4328-8ea2-3b1710a0198d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:57.449600 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc88d378-cd38-4328-8ea2-3b1710a0198d admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:57.449890 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc88d378-cd38-4328-8ea2-3b1710a0198d admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:57.450188 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc88d378-cd38-4328-8ea2-3b1710a0198d admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:57.450929 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc88d378-cd38-4328-8ea2-3b1710a0198d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:57.452570 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc88d378-cd38-4328-8ea2-3b1710a0198d admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:57.464696 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1020/2028] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:37:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:37:57.470026 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91109f12-0ed5-4f89-9641-4d8be8a0c80b admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:57.485352 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91109f12-0ed5-4f89-9641-4d8be8a0c80b admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:57.490034 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d667da1f-1df4-4735-a332-6ea8a0ea2100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:57.490925 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d667da1f-1df4-4735-a332-6ea8a0ea2100 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:57.491214 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d667da1f-1df4-4735-a332-6ea8a0ea2100 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:57.491494 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d667da1f-1df4-4735-a332-6ea8a0ea2100 admin admin] PATH_INFO: `/v3/projects/bd72f4cdad2f403b8492a2ae20e8eed0/users/1ea1a12e171641adbc362b30c96a5021/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:57.507058 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d667da1f-1df4-4735-a332-6ea8a0ea2100 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd72f4cdad2f403b8492a2ae20e8eed0, user_id=1ea1a12e171641adbc362b30c96a5021, role_id=d0f6ba1314f14d9f9c816bdb5d0c3aac)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:57.509966 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d667da1f-1df4-4735-a332-6ea8a0ea2100 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:57.524236 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1009/2029] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 15:37:57 2022] PUT /identity/v3/projects/bd72f4cdad2f403b8492a2ae20e8eed0/users/1ea1a12e171641adbc362b30c96a5021/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:37:57.528453 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc88d378-cd38-4328-8ea2-3b1710a0198d admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:57.539840 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc88d378-cd38-4328-8ea2-3b1710a0198d admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:57.590200 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:37:57.602837 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf6cb2b-bfa5-4a04-aab8-788c08a9cc83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:57.603511 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf6cb2b-bfa5-4a04-aab8-788c08a9cc83 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:57.603830 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf6cb2b-bfa5-4a04-aab8-788c08a9cc83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:57.604112 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf6cb2b-bfa5-4a04-aab8-788c08a9cc83 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:57.604944 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf6cb2b-bfa5-4a04-aab8-788c08a9cc83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:57.606549 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf6cb2b-bfa5-4a04-aab8-788c08a9cc83 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:57.617657 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1021/2030] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:37:57 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 15:37:57.621761 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d667da1f-1df4-4735-a332-6ea8a0ea2100 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:57.632389 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d667da1f-1df4-4735-a332-6ea8a0ea2100 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:57.637365 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d6f8a6-a11d-4583-a99d-c9d6c6cec394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:57.638282 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d6f8a6-a11d-4583-a99d-c9d6c6cec394 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:57.638560 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d6f8a6-a11d-4583-a99d-c9d6c6cec394 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:57.638835 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d6f8a6-a11d-4583-a99d-c9d6c6cec394 admin admin] PATH_INFO: `/v3/projects/bd72f4cdad2f403b8492a2ae20e8eed0/users/1ea1a12e171641adbc362b30c96a5021/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:57.641435 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d6f8a6-a11d-4583-a99d-c9d6c6cec394 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd72f4cdad2f403b8492a2ae20e8eed0, user_id=1ea1a12e171641adbc362b30c96a5021, role_id=d0f6ba1314f14d9f9c816bdb5d0c3aac)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:57.643545 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d6f8a6-a11d-4583-a99d-c9d6c6cec394 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:57.655124 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1010/2031] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 15:37:57 2022] PUT /identity/v3/projects/bd72f4cdad2f403b8492a2ae20e8eed0/users/1ea1a12e171641adbc362b30c96a5021/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 15:37:57.666356 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd7096d-9a31-464d-bbf9-0202a71b1f8f None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:57.666653 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd7096d-9a31-464d-bbf9-0202a71b1f8f None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:57.666934 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd7096d-9a31-464d-bbf9-0202a71b1f8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:57.693889 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-8dd7096d-9a31-464d-bbf9-0202a71b1f8f None None] MFA Rules not processed for user `1ea1a12e171641adbc362b30c96a5021`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:37:57.695094 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-8dd7096d-9a31-464d-bbf9-0202a71b1f8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:57.767920 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1022/2032] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:37:57 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:37:57.781785 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d6f8a6-a11d-4583-a99d-c9d6c6cec394 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:57.783632 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-14d6f8a6-a11d-4583-a99d-c9d6c6cec394 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:57.845773 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d6f8a6-a11d-4583-a99d-c9d6c6cec394 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:57.849530 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c706ea09-2a33-4e2f-a0df-5e6a7f6c1594 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3e1a2b83f3d4d88ba5084dfaffd22f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:57.850190 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c706ea09-2a33-4e2f-a0df-5e6a7f6c1594 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:57.850482 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c706ea09-2a33-4e2f-a0df-5e6a7f6c1594 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:57.850771 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c706ea09-2a33-4e2f-a0df-5e6a7f6c1594 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:57.859967 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c706ea09-2a33-4e2f-a0df-5e6a7f6c1594 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:57.861405 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c706ea09-2a33-4e2f-a0df-5e6a7f6c1594 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:57.870852 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1011/2033] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:37:57 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 15:37:57.942943 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd7096d-9a31-464d-bbf9-0202a71b1f8f None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:57.953302 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd7096d-9a31-464d-bbf9-0202a71b1f8f None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:57.958119 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060ea8c0-4fab-44bb-9651-18ee35440dc5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3e1a2b83f3d4d88ba5084dfaffd22f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:57.958780 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060ea8c0-4fab-44bb-9651-18ee35440dc5 service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:57.959068 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060ea8c0-4fab-44bb-9651-18ee35440dc5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:57.959376 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060ea8c0-4fab-44bb-9651-18ee35440dc5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:58.020179 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060ea8c0-4fab-44bb-9651-18ee35440dc5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:58.021892 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060ea8c0-4fab-44bb-9651-18ee35440dc5 service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:58.031220 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1023/2034] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:37:57 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:37:58.301636 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c706ea09-2a33-4e2f-a0df-5e6a7f6c1594 service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:58.314322 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:37:58.365503 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c706ea09-2a33-4e2f-a0df-5e6a7f6c1594 service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:58.370121 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78814cb5-5f01-4ea9-bd4a-3c7eacf001be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:58.371022 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78814cb5-5f01-4ea9-bd4a-3c7eacf001be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:58.371308 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78814cb5-5f01-4ea9-bd4a-3c7eacf001be admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:58.371585 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78814cb5-5f01-4ea9-bd4a-3c7eacf001be admin admin] PATH_INFO: `/v3/users/281a4bcae25349f89bf92234410c45a7` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:58.372797 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78814cb5-5f01-4ea9-bd4a-3c7eacf001be admin admin] RBAC: Authorizing `identity:delete_user(user_id=281a4bcae25349f89bf92234410c45a7)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:58.375422 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78814cb5-5f01-4ea9-bd4a-3c7eacf001be admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:58.424056 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-78814cb5-5f01-4ea9-bd4a-3c7eacf001be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '281a4bcae25349f89bf92234410c45a7'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:58.431357 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-78814cb5-5f01-4ea9-bd4a-3c7eacf001be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '281a4bcae25349f89bf92234410c45a7'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:58.444274 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-78814cb5-5f01-4ea9-bd4a-3c7eacf001be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '281a4bcae25349f89bf92234410c45a7'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:58.444915 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-78814cb5-5f01-4ea9-bd4a-3c7eacf001be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '281a4bcae25349f89bf92234410c45a7'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:58.445515 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-78814cb5-5f01-4ea9-bd4a-3c7eacf001be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '281a4bcae25349f89bf92234410c45a7'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:58.458037 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1012/2035] 10.222.0.99 () {58 vars in 1223 bytes} [Thu Jul 14 15:37:58 2022] DELETE /identity/v3/users/281a4bcae25349f89bf92234410c45a7 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:37:58.462559 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a993dfb1-e303-47d5-aa07-ddb876a90a09 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:58.462853 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a993dfb1-e303-47d5-aa07-ddb876a90a09 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:58.463131 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a993dfb1-e303-47d5-aa07-ddb876a90a09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:58.480178 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-a993dfb1-e303-47d5-aa07-ddb876a90a09 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:37:58.481453 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-a993dfb1-e303-47d5-aa07-ddb876a90a09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:58.545572 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1024/2036] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:37:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 15:37:58.561094 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78814cb5-5f01-4ea9-bd4a-3c7eacf001be admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:58.568362 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-78814cb5-5f01-4ea9-bd4a-3c7eacf001be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:58.644421 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78814cb5-5f01-4ea9-bd4a-3c7eacf001be admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:58.648397 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c5d089-7dd5-462e-ac33-3b0d34136171 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:58.649065 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c5d089-7dd5-462e-ac33-3b0d34136171 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:58.649356 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c5d089-7dd5-462e-ac33-3b0d34136171 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:58.649642 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c5d089-7dd5-462e-ac33-3b0d34136171 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:58.659851 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c5d089-7dd5-462e-ac33-3b0d34136171 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:58.661294 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c5d089-7dd5-462e-ac33-3b0d34136171 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:58.672303 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1013/2037] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:37:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:37:59.227542 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a993dfb1-e303-47d5-aa07-ddb876a90a09 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:59.231153 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-a993dfb1-e303-47d5-aa07-ddb876a90a09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:59.260581 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a993dfb1-e303-47d5-aa07-ddb876a90a09 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:59.266998 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc4edc8-a36a-4ea5-89e6-7cd0c71919bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:59.268081 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc4edc8-a36a-4ea5-89e6-7cd0c71919bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:59.269438 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc4edc8-a36a-4ea5-89e6-7cd0c71919bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:59.269841 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc4edc8-a36a-4ea5-89e6-7cd0c71919bb admin admin] PATH_INFO: `/v3/projects/6f3ed484beff472d899c8e13f263f426` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:59.271487 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc4edc8-a36a-4ea5-89e6-7cd0c71919bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f3ed484beff472d899c8e13f263f426)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:59.274896 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc4edc8-a36a-4ea5-89e6-7cd0c71919bb admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:59.284911 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:37:59.327049 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-ffc4edc8-a36a-4ea5-89e6-7cd0c71919bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f3ed484beff472d899c8e13f263f426', 'request_id': 'req-ffc4edc8-a36a-4ea5-89e6-7cd0c71919bb'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:59.335083 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-ffc4edc8-a36a-4ea5-89e6-7cd0c71919bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f3ed484beff472d899c8e13f263f426', 'request_id': 'req-ffc4edc8-a36a-4ea5-89e6-7cd0c71919bb'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:59.341888 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-ffc4edc8-a36a-4ea5-89e6-7cd0c71919bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f3ed484beff472d899c8e13f263f426', 'request_id': 'req-ffc4edc8-a36a-4ea5-89e6-7cd0c71919bb'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:59.357846 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-ffc4edc8-a36a-4ea5-89e6-7cd0c71919bb admin admin] The token cache is being invalidate because project 6f3ed484beff472d899c8e13f263f426 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 15:37:59.358138 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-ffc4edc8-a36a-4ea5-89e6-7cd0c71919bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:59.358753 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-ffc4edc8-a36a-4ea5-89e6-7cd0c71919bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:59.361017 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1025/2038] 10.222.0.99 () {58 vars in 1232 bytes} [Thu Jul 14 15:37:59 2022] DELETE /identity/v3/projects/6f3ed484beff472d899c8e13f263f426 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:37:59.365062 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c5d089-7dd5-462e-ac33-3b0d34136171 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:59.375531 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c5d089-7dd5-462e-ac33-3b0d34136171 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:59.419695 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:37:59.454562 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0991b85-09f0-4b49-8280-c2fb733128b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:59.455362 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0991b85-09f0-4b49-8280-c2fb733128b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:59.455695 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0991b85-09f0-4b49-8280-c2fb733128b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:59.455980 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0991b85-09f0-4b49-8280-c2fb733128b8 admin admin] PATH_INFO: `/v3/users/1ea1a12e171641adbc362b30c96a5021` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:59.457206 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0991b85-09f0-4b49-8280-c2fb733128b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ea1a12e171641adbc362b30c96a5021)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:59.459144 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0991b85-09f0-4b49-8280-c2fb733128b8 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:59.541862 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d0991b85-09f0-4b49-8280-c2fb733128b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ea1a12e171641adbc362b30c96a5021'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:59.553251 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d0991b85-09f0-4b49-8280-c2fb733128b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ea1a12e171641adbc362b30c96a5021'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:59.572649 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d0991b85-09f0-4b49-8280-c2fb733128b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ea1a12e171641adbc362b30c96a5021'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:59.573497 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d0991b85-09f0-4b49-8280-c2fb733128b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ea1a12e171641adbc362b30c96a5021'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:59.574120 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d0991b85-09f0-4b49-8280-c2fb733128b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ea1a12e171641adbc362b30c96a5021'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:37:59.590203 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1014/2039] 10.222.0.99 () {58 vars in 1223 bytes} [Thu Jul 14 15:37:59 2022] DELETE /identity/v3/users/1ea1a12e171641adbc362b30c96a5021 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:38:00.298532 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc4edc8-a36a-4ea5-89e6-7cd0c71919bb admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:00.301568 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-ffc4edc8-a36a-4ea5-89e6-7cd0c71919bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:00.314842 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc4edc8-a36a-4ea5-89e6-7cd0c71919bb admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:00.381331 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b8bf08-1045-448f-9cab-7feed0f387ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:00.382290 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b8bf08-1045-448f-9cab-7feed0f387ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:00.382652 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b8bf08-1045-448f-9cab-7feed0f387ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:00.383009 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b8bf08-1045-448f-9cab-7feed0f387ed admin admin] PATH_INFO: `/v3/projects/bd72f4cdad2f403b8492a2ae20e8eed0` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:00.384343 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b8bf08-1045-448f-9cab-7feed0f387ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd72f4cdad2f403b8492a2ae20e8eed0)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:00.386597 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b8bf08-1045-448f-9cab-7feed0f387ed admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:00.444504 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-06b8bf08-1045-448f-9cab-7feed0f387ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd72f4cdad2f403b8492a2ae20e8eed0', 'request_id': 'req-06b8bf08-1045-448f-9cab-7feed0f387ed'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:00.450891 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-06b8bf08-1045-448f-9cab-7feed0f387ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd72f4cdad2f403b8492a2ae20e8eed0', 'request_id': 'req-06b8bf08-1045-448f-9cab-7feed0f387ed'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:00.467668 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-06b8bf08-1045-448f-9cab-7feed0f387ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd72f4cdad2f403b8492a2ae20e8eed0', 'request_id': 'req-06b8bf08-1045-448f-9cab-7feed0f387ed'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:00.480846 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-06b8bf08-1045-448f-9cab-7feed0f387ed admin admin] The token cache is being invalidate because project bd72f4cdad2f403b8492a2ae20e8eed0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 15:38:00.481243 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-06b8bf08-1045-448f-9cab-7feed0f387ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:00.481913 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-06b8bf08-1045-448f-9cab-7feed0f387ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:00.483506 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1026/2040] 10.222.0.99 () {58 vars in 1232 bytes} [Thu Jul 14 15:38:00 2022] DELETE /identity/v3/projects/bd72f4cdad2f403b8492a2ae20e8eed0 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:38:00.489088 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de6b367-ce9e-48c6-856e-628246a3b0aa None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:00.489738 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de6b367-ce9e-48c6-856e-628246a3b0aa None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:00.490040 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de6b367-ce9e-48c6-856e-628246a3b0aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:00.509638 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-5de6b367-ce9e-48c6-856e-628246a3b0aa None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:38:00.510874 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-5de6b367-ce9e-48c6-856e-628246a3b0aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:00.577119 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1015/2041] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:38:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:38:00.585652 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf9e81f-fad2-4c07-9436-c71c8e6cd744 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:00.585948 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf9e81f-fad2-4c07-9436-c71c8e6cd744 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:00.586232 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf9e81f-fad2-4c07-9436-c71c8e6cd744 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:00.605736 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-5cf9e81f-fad2-4c07-9436-c71c8e6cd744 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:38:00.606947 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-5cf9e81f-fad2-4c07-9436-c71c8e6cd744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:00.615726 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1027/2042] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:38:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:38:00.619960 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de6b367-ce9e-48c6-856e-628246a3b0aa None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:00.630062 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de6b367-ce9e-48c6-856e-628246a3b0aa None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:00.634802 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fb8557-a1d9-47e1-9bd0-70eac327bad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:00.635489 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fb8557-a1d9-47e1-9bd0-70eac327bad8 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:00.635813 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fb8557-a1d9-47e1-9bd0-70eac327bad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:00.636101 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fb8557-a1d9-47e1-9bd0-70eac327bad8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:00.636841 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fb8557-a1d9-47e1-9bd0-70eac327bad8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:38:00.637194 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fb8557-a1d9-47e1-9bd0-70eac327bad8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:00.638844 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fb8557-a1d9-47e1-9bd0-70eac327bad8 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:00.653012 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1016/2043] 10.222.0.99 () {58 vars in 1152 bytes} [Thu Jul 14 15:38:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:38:00.658148 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf9e81f-fad2-4c07-9436-c71c8e6cd744 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:00.669678 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf9e81f-fad2-4c07-9436-c71c8e6cd744 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:00.676530 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286ee28a-365a-41c0-9964-fd59d305a555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:00.677220 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286ee28a-365a-41c0-9964-fd59d305a555 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:00.677537 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286ee28a-365a-41c0-9964-fd59d305a555 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:00.677923 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286ee28a-365a-41c0-9964-fd59d305a555 admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:00.678652 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286ee28a-365a-41c0-9964-fd59d305a555 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:00.680676 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286ee28a-365a-41c0-9964-fd59d305a555 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:00.708605 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:00.718566 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1028/2044] 10.222.0.99 () {60 vars in 1152 bytes} [Thu Jul 14 15:38:00 2022] POST /identity/v3/projects => generated 381 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 15:38:00.723172 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fb8557-a1d9-47e1-9bd0-70eac327bad8 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:00.736099 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fb8557-a1d9-47e1-9bd0-70eac327bad8 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:00.788394 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:00.814393 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f411e9ca-836f-4df9-95d6-d62e2d806efb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:00.815083 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f411e9ca-836f-4df9-95d6-d62e2d806efb admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:00.815370 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f411e9ca-836f-4df9-95d6-d62e2d806efb admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:00.815690 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f411e9ca-836f-4df9-95d6-d62e2d806efb admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:00.816448 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f411e9ca-836f-4df9-95d6-d62e2d806efb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:00.818499 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f411e9ca-836f-4df9-95d6-d62e2d806efb admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:00.847498 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1017/2045] 10.222.0.99 () {60 vars in 1143 bytes} [Thu Jul 14 15:38:00 2022] POST /identity/v3/users => generated 291 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:38:00.851698 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286ee28a-365a-41c0-9964-fd59d305a555 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:00.866483 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286ee28a-365a-41c0-9964-fd59d305a555 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:00.882783 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52b4b52-4f03-4a9f-a9c0-7ff774fea18b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:00.883824 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52b4b52-4f03-4a9f-a9c0-7ff774fea18b admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:00.884328 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52b4b52-4f03-4a9f-a9c0-7ff774fea18b admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:00.884756 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52b4b52-4f03-4a9f-a9c0-7ff774fea18b admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:00.885720 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52b4b52-4f03-4a9f-a9c0-7ff774fea18b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:00.887499 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52b4b52-4f03-4a9f-a9c0-7ff774fea18b admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:00.912185 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1029/2046] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:38:00 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:38:00.914124 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f411e9ca-836f-4df9-95d6-d62e2d806efb admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:00.930025 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f411e9ca-836f-4df9-95d6-d62e2d806efb admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:00.946618 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c363c69b-7409-419d-a40e-29bf053d37b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:00.947696 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c363c69b-7409-419d-a40e-29bf053d37b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:00.949339 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c363c69b-7409-419d-a40e-29bf053d37b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:00.949916 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c363c69b-7409-419d-a40e-29bf053d37b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:00.950836 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c363c69b-7409-419d-a40e-29bf053d37b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:00.958929 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c363c69b-7409-419d-a40e-29bf053d37b4 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:00.981718 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1018/2047] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:38:00 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:38:00.986878 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52b4b52-4f03-4a9f-a9c0-7ff774fea18b admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:01.011106 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52b4b52-4f03-4a9f-a9c0-7ff774fea18b admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:01.028788 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8dcdff-11d2-49f0-a94c-0dc2b1b7911a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:01.030123 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8dcdff-11d2-49f0-a94c-0dc2b1b7911a admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:01.030422 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8dcdff-11d2-49f0-a94c-0dc2b1b7911a admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:01.030707 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8dcdff-11d2-49f0-a94c-0dc2b1b7911a admin admin] PATH_INFO: `/v3/projects/523e4517a98c4067a338372c714d7c01/users/89f16191fe98468cb2798246f4475ea1/roles/b8c98fddc0fd4436b577fd1d3361bc38` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:01.070918 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8dcdff-11d2-49f0-a94c-0dc2b1b7911a admin admin] RBAC: Authorizing `identity:create_grant(project_id=523e4517a98c4067a338372c714d7c01, user_id=89f16191fe98468cb2798246f4475ea1, role_id=b8c98fddc0fd4436b577fd1d3361bc38)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:01.073894 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8dcdff-11d2-49f0-a94c-0dc2b1b7911a admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:01.107973 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1030/2048] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 15:38:00 2022] PUT /identity/v3/projects/523e4517a98c4067a338372c714d7c01/users/89f16191fe98468cb2798246f4475ea1/roles/b8c98fddc0fd4436b577fd1d3361bc38 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:38:01.118470 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6edebbc-40e2-42af-9525-e7f13886a5b5 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:01.119126 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6edebbc-40e2-42af-9525-e7f13886a5b5 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:01.119491 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6edebbc-40e2-42af-9525-e7f13886a5b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:01.208112 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-e6edebbc-40e2-42af-9525-e7f13886a5b5 None None] MFA Rules not processed for user `89f16191fe98468cb2798246f4475ea1`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:38:01.209970 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e6edebbc-40e2-42af-9525-e7f13886a5b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:01.314661 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1019/2049] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:38:01 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:38:01.364904 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8dcdff-11d2-49f0-a94c-0dc2b1b7911a admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:01.367847 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-6d8dcdff-11d2-49f0-a94c-0dc2b1b7911a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:01.444052 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8dcdff-11d2-49f0-a94c-0dc2b1b7911a admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:01.448594 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf065fd-c46c-48ce-92b1-1cb0a710937a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3e1a2b83f3d4d88ba5084dfaffd22f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:01.449483 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf065fd-c46c-48ce-92b1-1cb0a710937a service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:01.449950 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf065fd-c46c-48ce-92b1-1cb0a710937a service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:01.450753 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf065fd-c46c-48ce-92b1-1cb0a710937a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:01.464434 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf065fd-c46c-48ce-92b1-1cb0a710937a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:01.466619 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf065fd-c46c-48ce-92b1-1cb0a710937a service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:01.486889 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1031/2050] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:38:01 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:38:01.517632 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6edebbc-40e2-42af-9525-e7f13886a5b5 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:01.633883 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6edebbc-40e2-42af-9525-e7f13886a5b5 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:01.639624 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8fe655-974c-42e9-9370-b5603bd11556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:01.641374 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8fe655-974c-42e9-9370-b5603bd11556 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:01.641675 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8fe655-974c-42e9-9370-b5603bd11556 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:01.641938 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8fe655-974c-42e9-9370-b5603bd11556 admin admin] PATH_INFO: `/v3/users/89f16191fe98468cb2798246f4475ea1` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:01.643342 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8fe655-974c-42e9-9370-b5603bd11556 admin admin] RBAC: Authorizing `identity:delete_user(user_id=89f16191fe98468cb2798246f4475ea1)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:01.647142 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8fe655-974c-42e9-9370-b5603bd11556 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:01.726513 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-ba8fe655-974c-42e9-9370-b5603bd11556 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89f16191fe98468cb2798246f4475ea1'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:01.731288 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:01.737527 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-ba8fe655-974c-42e9-9370-b5603bd11556 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89f16191fe98468cb2798246f4475ea1'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:01.756145 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-ba8fe655-974c-42e9-9370-b5603bd11556 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89f16191fe98468cb2798246f4475ea1'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:01.756903 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-ba8fe655-974c-42e9-9370-b5603bd11556 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89f16191fe98468cb2798246f4475ea1'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:01.757537 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-ba8fe655-974c-42e9-9370-b5603bd11556 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89f16191fe98468cb2798246f4475ea1'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:01.776902 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1020/2051] 10.222.0.99 () {58 vars in 1223 bytes} [Thu Jul 14 15:38:01 2022] DELETE /identity/v3/users/89f16191fe98468cb2798246f4475ea1 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:38:01.783789 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61a438a-99a5-4904-a927-1d6c1d861964 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:01.784269 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61a438a-99a5-4904-a927-1d6c1d861964 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:01.784967 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61a438a-99a5-4904-a927-1d6c1d861964 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:01.805886 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-b61a438a-99a5-4904-a927-1d6c1d861964 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:38:01.807215 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-b61a438a-99a5-4904-a927-1d6c1d861964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:01.872211 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1032/2052] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:38:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:38:01.882336 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8fe655-974c-42e9-9370-b5603bd11556 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:01.883827 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ba8fe655-974c-42e9-9370-b5603bd11556 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:01.949004 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8fe655-974c-42e9-9370-b5603bd11556 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:01.952650 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df778c24-5399-4950-9c41-05dcae9b2b0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:01.953978 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df778c24-5399-4950-9c41-05dcae9b2b0d service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:01.954260 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df778c24-5399-4950-9c41-05dcae9b2b0d service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:01.954533 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df778c24-5399-4950-9c41-05dcae9b2b0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:01.964932 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df778c24-5399-4950-9c41-05dcae9b2b0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:01.966377 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df778c24-5399-4950-9c41-05dcae9b2b0d service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:01.976774 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1021/2053] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:38:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:38:02.272566 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61a438a-99a5-4904-a927-1d6c1d861964 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:02.274360 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-b61a438a-99a5-4904-a927-1d6c1d861964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:02.288415 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61a438a-99a5-4904-a927-1d6c1d861964 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:02.295957 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26e0f75-1699-4b2b-b53a-7f42a5cc9fa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:02.297549 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26e0f75-1699-4b2b-b53a-7f42a5cc9fa7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:02.297785 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26e0f75-1699-4b2b-b53a-7f42a5cc9fa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:02.298066 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26e0f75-1699-4b2b-b53a-7f42a5cc9fa7 admin admin] PATH_INFO: `/v3/projects/523e4517a98c4067a338372c714d7c01` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:02.299400 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26e0f75-1699-4b2b-b53a-7f42a5cc9fa7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=523e4517a98c4067a338372c714d7c01)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:02.301378 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26e0f75-1699-4b2b-b53a-7f42a5cc9fa7 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:02.358450 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-e26e0f75-1699-4b2b-b53a-7f42a5cc9fa7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '523e4517a98c4067a338372c714d7c01', 'request_id': 'req-e26e0f75-1699-4b2b-b53a-7f42a5cc9fa7'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:02.365988 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-e26e0f75-1699-4b2b-b53a-7f42a5cc9fa7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '523e4517a98c4067a338372c714d7c01', 'request_id': 'req-e26e0f75-1699-4b2b-b53a-7f42a5cc9fa7'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:02.371099 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:02.372429 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-e26e0f75-1699-4b2b-b53a-7f42a5cc9fa7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '523e4517a98c4067a338372c714d7c01', 'request_id': 'req-e26e0f75-1699-4b2b-b53a-7f42a5cc9fa7'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:02.391405 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-e26e0f75-1699-4b2b-b53a-7f42a5cc9fa7 admin admin] The token cache is being invalidate because project 523e4517a98c4067a338372c714d7c01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 15:38:02.392540 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-e26e0f75-1699-4b2b-b53a-7f42a5cc9fa7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:02.393929 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-e26e0f75-1699-4b2b-b53a-7f42a5cc9fa7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:02.395925 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1033/2054] 10.222.0.99 () {58 vars in 1232 bytes} [Thu Jul 14 15:38:02 2022] DELETE /identity/v3/projects/523e4517a98c4067a338372c714d7c01 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 15:38:02.401325 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fb1bcb-92e7-4e38-ae83-5b319a606301 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:02.401700 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fb1bcb-92e7-4e38-ae83-5b319a606301 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:02.402052 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fb1bcb-92e7-4e38-ae83-5b319a606301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:02.420705 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-80fb1bcb-92e7-4e38-ae83-5b319a606301 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:38:02.422771 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-80fb1bcb-92e7-4e38-ae83-5b319a606301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:02.498950 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1022/2055] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:38:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 15:38:02.508418 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec233c2-38e0-4f56-b094-31c0a13e58b3 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:02.508721 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec233c2-38e0-4f56-b094-31c0a13e58b3 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:02.509018 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec233c2-38e0-4f56-b094-31c0a13e58b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:02.529631 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-3ec233c2-38e0-4f56-b094-31c0a13e58b3 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:38:02.531634 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-3ec233c2-38e0-4f56-b094-31c0a13e58b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:02.540914 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1034/2056] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:38:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:38:02.551107 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fb1bcb-92e7-4e38-ae83-5b319a606301 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:02.564663 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fb1bcb-92e7-4e38-ae83-5b319a606301 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:02.569774 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88ab057-1f57-4b7b-ac69-eec81613e12e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:02.570588 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88ab057-1f57-4b7b-ac69-eec81613e12e admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:02.570953 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88ab057-1f57-4b7b-ac69-eec81613e12e admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:02.571296 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88ab057-1f57-4b7b-ac69-eec81613e12e admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:02.572144 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88ab057-1f57-4b7b-ac69-eec81613e12e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:38:02.572523 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88ab057-1f57-4b7b-ac69-eec81613e12e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:02.574319 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88ab057-1f57-4b7b-ac69-eec81613e12e admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:02.588097 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1023/2057] 10.222.0.99 () {58 vars in 1152 bytes} [Thu Jul 14 15:38:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:38:02.592459 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec233c2-38e0-4f56-b094-31c0a13e58b3 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:02.603466 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec233c2-38e0-4f56-b094-31c0a13e58b3 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:02.608295 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06567a39-1498-4a11-9975-8be1c8e679f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:02.609042 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06567a39-1498-4a11-9975-8be1c8e679f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:02.610040 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06567a39-1498-4a11-9975-8be1c8e679f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:02.610040 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06567a39-1498-4a11-9975-8be1c8e679f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:02.611055 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06567a39-1498-4a11-9975-8be1c8e679f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:02.613239 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06567a39-1498-4a11-9975-8be1c8e679f4 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:02.641913 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1035/2058] 10.222.0.99 () {60 vars in 1152 bytes} [Thu Jul 14 15:38:02 2022] POST /identity/v3/projects => generated 375 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:38:02.646665 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88ab057-1f57-4b7b-ac69-eec81613e12e admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:02.658032 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88ab057-1f57-4b7b-ac69-eec81613e12e admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:02.712437 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:DELETE=3,keystone:SELECT=97 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:02.720141 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b1a206-4a48-4a6c-a781-d5bc92e9e677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:02.720878 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b1a206-4a48-4a6c-a781-d5bc92e9e677 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:02.722603 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b1a206-4a48-4a6c-a781-d5bc92e9e677 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:02.722603 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b1a206-4a48-4a6c-a781-d5bc92e9e677 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:02.722603 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b1a206-4a48-4a6c-a781-d5bc92e9e677 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:02.724678 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b1a206-4a48-4a6c-a781-d5bc92e9e677 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:02.748290 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1024/2059] 10.222.0.99 () {60 vars in 1143 bytes} [Thu Jul 14 15:38:02 2022] POST /identity/v3/users => generated 288 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 15:38:02.749515 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06567a39-1498-4a11-9975-8be1c8e679f4 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:02.761009 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06567a39-1498-4a11-9975-8be1c8e679f4 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:02.775664 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caca06f9-7b7c-419a-b362-1c897a61e176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:02.776370 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caca06f9-7b7c-419a-b362-1c897a61e176 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:02.776658 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caca06f9-7b7c-419a-b362-1c897a61e176 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:02.776939 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caca06f9-7b7c-419a-b362-1c897a61e176 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:02.777685 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caca06f9-7b7c-419a-b362-1c897a61e176 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:02.779475 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caca06f9-7b7c-419a-b362-1c897a61e176 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:02.793295 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1036/2060] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:38:02 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 15:38:02.797854 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b1a206-4a48-4a6c-a781-d5bc92e9e677 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:02.812503 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b1a206-4a48-4a6c-a781-d5bc92e9e677 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:02.819299 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bad9d25-2e21-492e-bff8-30829e3b9d5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:02.820017 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bad9d25-2e21-492e-bff8-30829e3b9d5c admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:02.820310 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bad9d25-2e21-492e-bff8-30829e3b9d5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:02.820594 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bad9d25-2e21-492e-bff8-30829e3b9d5c admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:02.821335 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bad9d25-2e21-492e-bff8-30829e3b9d5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:02.822944 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bad9d25-2e21-492e-bff8-30829e3b9d5c admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:02.837643 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1025/2061] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:38:02 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:38:02.842294 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caca06f9-7b7c-419a-b362-1c897a61e176 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:02.855091 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caca06f9-7b7c-419a-b362-1c897a61e176 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:02.862339 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0e1941-23da-49b3-927b-918f5ac6e124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:02.863252 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0e1941-23da-49b3-927b-918f5ac6e124 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:02.863537 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0e1941-23da-49b3-927b-918f5ac6e124 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:02.863850 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0e1941-23da-49b3-927b-918f5ac6e124 admin admin] PATH_INFO: `/v3/projects/a6fb7d0070ab49f1a4e8224179d09e4b/users/f69abc1537be4ca99a2741ddbaca6faa/roles/b8c98fddc0fd4436b577fd1d3361bc38` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:02.885275 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0e1941-23da-49b3-927b-918f5ac6e124 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6fb7d0070ab49f1a4e8224179d09e4b, user_id=f69abc1537be4ca99a2741ddbaca6faa, role_id=b8c98fddc0fd4436b577fd1d3361bc38)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:02.888165 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0e1941-23da-49b3-927b-918f5ac6e124 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:02.907448 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1037/2062] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 15:38:02 2022] PUT /identity/v3/projects/a6fb7d0070ab49f1a4e8224179d09e4b/users/f69abc1537be4ca99a2741ddbaca6faa/roles/b8c98fddc0fd4436b577fd1d3361bc38 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 15:38:02.917528 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c1c4a7-5041-440b-adfd-63c26ff566ca None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:02.917851 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c1c4a7-5041-440b-adfd-63c26ff566ca None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:02.918133 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c1c4a7-5041-440b-adfd-63c26ff566ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:02.951268 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-a1c1c4a7-5041-440b-adfd-63c26ff566ca None None] MFA Rules not processed for user `f69abc1537be4ca99a2741ddbaca6faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:38:02.952722 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a1c1c4a7-5041-440b-adfd-63c26ff566ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:03.038719 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1026/2063] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:38:02 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:38:03.050438 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0e1941-23da-49b3-927b-918f5ac6e124 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:03.052291 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-5a0e1941-23da-49b3-927b-918f5ac6e124 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:03.144585 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0e1941-23da-49b3-927b-918f5ac6e124 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:03.154096 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6c7bce-81e3-4af9-a886-a58153cbd3c1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605e230d50ce44629f554c95b2f79ead', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:03.154842 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6c7bce-81e3-4af9-a886-a58153cbd3c1 service glance] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:03.155240 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6c7bce-81e3-4af9-a886-a58153cbd3c1 service glance] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:03.155657 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6c7bce-81e3-4af9-a886-a58153cbd3c1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:03.166048 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6c7bce-81e3-4af9-a886-a58153cbd3c1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:03.167676 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6c7bce-81e3-4af9-a886-a58153cbd3c1 service glance] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:03.178304 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1038/2064] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:38:03 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:38:10.858596 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c1c4a7-5041-440b-adfd-63c26ff566ca None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:10.861704 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a1c1c4a7-5041-440b-adfd-63c26ff566ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:10.943193 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c1c4a7-5041-440b-adfd-63c26ff566ca None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:10.948656 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd2ea89-fce4-418c-a5ec-b5e9ed86ff91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:10.949895 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd2ea89-fce4-418c-a5ec-b5e9ed86ff91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:10.950380 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd2ea89-fce4-418c-a5ec-b5e9ed86ff91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:10.951032 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd2ea89-fce4-418c-a5ec-b5e9ed86ff91 admin admin] PATH_INFO: `/v3/users/b04c8de464b14622ba4a4a3b5e7c3658` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:10.952668 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd2ea89-fce4-418c-a5ec-b5e9ed86ff91 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b04c8de464b14622ba4a4a3b5e7c3658)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:10.955841 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd2ea89-fce4-418c-a5ec-b5e9ed86ff91 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:11.032428 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-6cd2ea89-fce4-418c-a5ec-b5e9ed86ff91 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b04c8de464b14622ba4a4a3b5e7c3658'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:11.042952 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-6cd2ea89-fce4-418c-a5ec-b5e9ed86ff91 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b04c8de464b14622ba4a4a3b5e7c3658'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:11.060277 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=87,keystone:INSERT=4,keystone:DELETE=9 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:11.064912 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-6cd2ea89-fce4-418c-a5ec-b5e9ed86ff91 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b04c8de464b14622ba4a4a3b5e7c3658'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:11.073300 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-6cd2ea89-fce4-418c-a5ec-b5e9ed86ff91 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b04c8de464b14622ba4a4a3b5e7c3658'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:11.078309 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-6cd2ea89-fce4-418c-a5ec-b5e9ed86ff91 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b04c8de464b14622ba4a4a3b5e7c3658'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:11.079930 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6c7bce-81e3-4af9-a886-a58153cbd3c1 service glance] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:11.082127 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-dc6c7bce-81e3-4af9-a886-a58153cbd3c1 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:11.102615 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6c7bce-81e3-4af9-a886-a58153cbd3c1 service glance] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:11.111220 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cf2a12-7ae3-445a-bebc-bd3b6c09fe26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:11.112170 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cf2a12-7ae3-445a-bebc-bd3b6c09fe26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:11.112430 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cf2a12-7ae3-445a-bebc-bd3b6c09fe26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:11.112708 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cf2a12-7ae3-445a-bebc-bd3b6c09fe26 admin admin] PATH_INFO: `/v3/users/cf782fc2f7dd418a80d94b73532106b9` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:11.113947 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cf2a12-7ae3-445a-bebc-bd3b6c09fe26 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf782fc2f7dd418a80d94b73532106b9)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:11.115429 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1027/2065] 10.222.0.99 () {58 vars in 1223 bytes} [Thu Jul 14 15:38:10 2022] DELETE /identity/v3/users/b04c8de464b14622ba4a4a3b5e7c3658 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:38:11.115944 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cf2a12-7ae3-445a-bebc-bd3b6c09fe26 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:11.178811 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-05cf2a12-7ae3-445a-bebc-bd3b6c09fe26 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf782fc2f7dd418a80d94b73532106b9'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:11.191288 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-05cf2a12-7ae3-445a-bebc-bd3b6c09fe26 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf782fc2f7dd418a80d94b73532106b9'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:11.204124 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:11.215970 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-05cf2a12-7ae3-445a-bebc-bd3b6c09fe26 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf782fc2f7dd418a80d94b73532106b9'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:11.216811 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-05cf2a12-7ae3-445a-bebc-bd3b6c09fe26 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf782fc2f7dd418a80d94b73532106b9'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:11.221567 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-05cf2a12-7ae3-445a-bebc-bd3b6c09fe26 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf782fc2f7dd418a80d94b73532106b9'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:11.237874 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1039/2066] 10.222.0.99 () {58 vars in 1223 bytes} [Thu Jul 14 15:38:11 2022] DELETE /identity/v3/users/cf782fc2f7dd418a80d94b73532106b9 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:38:11.245728 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94097708-d686-4054-bca5-d3f835627ccb None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:11.247058 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94097708-d686-4054-bca5-d3f835627ccb None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:11.248140 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94097708-d686-4054-bca5-d3f835627ccb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:11.275635 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-94097708-d686-4054-bca5-d3f835627ccb None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:38:11.276775 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-94097708-d686-4054-bca5-d3f835627ccb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:11.386825 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1028/2067] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:38:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:38:11.401523 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cf2a12-7ae3-445a-bebc-bd3b6c09fe26 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:11.402985 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-05cf2a12-7ae3-445a-bebc-bd3b6c09fe26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:11.470301 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cf2a12-7ae3-445a-bebc-bd3b6c09fe26 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:11.474522 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d044809-3d22-4d37-8f1f-ca1fb724ec57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:11.475244 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d044809-3d22-4d37-8f1f-ca1fb724ec57 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:11.475715 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d044809-3d22-4d37-8f1f-ca1fb724ec57 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:11.476080 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d044809-3d22-4d37-8f1f-ca1fb724ec57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:11.488702 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d044809-3d22-4d37-8f1f-ca1fb724ec57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:11.490318 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d044809-3d22-4d37-8f1f-ca1fb724ec57 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:11.501139 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1040/2068] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:38:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:38:11.689133 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94097708-d686-4054-bca5-d3f835627ccb None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:11.698499 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-94097708-d686-4054-bca5-d3f835627ccb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:11.754991 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94097708-d686-4054-bca5-d3f835627ccb None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:11.766366 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0ba67d-b927-4343-8673-1666f5ef9e94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:11.768458 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0ba67d-b927-4343-8673-1666f5ef9e94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:11.768910 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0ba67d-b927-4343-8673-1666f5ef9e94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:11.769402 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0ba67d-b927-4343-8673-1666f5ef9e94 admin admin] PATH_INFO: `/v3/projects/ead3dc4d1dcf450098946be77432bce2` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:11.771063 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0ba67d-b927-4343-8673-1666f5ef9e94 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ead3dc4d1dcf450098946be77432bce2)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:11.773884 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0ba67d-b927-4343-8673-1666f5ef9e94 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:11.892880 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-dc0ba67d-b927-4343-8673-1666f5ef9e94 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ead3dc4d1dcf450098946be77432bce2', 'request_id': 'req-dc0ba67d-b927-4343-8673-1666f5ef9e94'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:11.900956 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-dc0ba67d-b927-4343-8673-1666f5ef9e94 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ead3dc4d1dcf450098946be77432bce2', 'request_id': 'req-dc0ba67d-b927-4343-8673-1666f5ef9e94'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:11.908347 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-dc0ba67d-b927-4343-8673-1666f5ef9e94 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ead3dc4d1dcf450098946be77432bce2', 'request_id': 'req-dc0ba67d-b927-4343-8673-1666f5ef9e94'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:11.917747 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d044809-3d22-4d37-8f1f-ca1fb724ec57 service neutron] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:11.919552 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-7d044809-3d22-4d37-8f1f-ca1fb724ec57 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:11.928103 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-dc0ba67d-b927-4343-8673-1666f5ef9e94 admin admin] The token cache is being invalidate because project ead3dc4d1dcf450098946be77432bce2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 15:38:11.928407 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-dc0ba67d-b927-4343-8673-1666f5ef9e94 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:11.929115 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-dc0ba67d-b927-4343-8673-1666f5ef9e94 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:11.931547 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1029/2069] 10.222.0.99 () {58 vars in 1232 bytes} [Thu Jul 14 15:38:11 2022] DELETE /identity/v3/projects/ead3dc4d1dcf450098946be77432bce2 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:38:11.935506 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0ba67d-b927-4343-8673-1666f5ef9e94 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:11.937214 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-dc0ba67d-b927-4343-8673-1666f5ef9e94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:11.947584 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d044809-3d22-4d37-8f1f-ca1fb724ec57 service neutron] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:12.052219 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59e92ce-fa0e-42f0-ac58-3774d58a64ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:12.053765 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59e92ce-fa0e-42f0-ac58-3774d58a64ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:12.054464 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59e92ce-fa0e-42f0-ac58-3774d58a64ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:12.054899 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59e92ce-fa0e-42f0-ac58-3774d58a64ba admin admin] PATH_INFO: `/v3/projects/e338799da18942aa89350a3b4cfba6a9` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:12.056701 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59e92ce-fa0e-42f0-ac58-3774d58a64ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=e338799da18942aa89350a3b4cfba6a9)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:12.060543 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59e92ce-fa0e-42f0-ac58-3774d58a64ba admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:12.081635 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0ba67d-b927-4343-8673-1666f5ef9e94 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:12.086862 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2f7aba-107e-4010-88a5-5c160493e260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:12.088044 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2f7aba-107e-4010-88a5-5c160493e260 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:12.088265 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2f7aba-107e-4010-88a5-5c160493e260 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:12.088544 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2f7aba-107e-4010-88a5-5c160493e260 admin admin] PATH_INFO: `/v3/users/70a38ee462734c6ab5c91aa3fbac669c` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:12.089811 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2f7aba-107e-4010-88a5-5c160493e260 admin admin] RBAC: Authorizing `identity:delete_user(user_id=70a38ee462734c6ab5c91aa3fbac669c)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:12.092666 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2f7aba-107e-4010-88a5-5c160493e260 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:12.118305 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:12.129735 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-f59e92ce-fa0e-42f0-ac58-3774d58a64ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e338799da18942aa89350a3b4cfba6a9', 'request_id': 'req-f59e92ce-fa0e-42f0-ac58-3774d58a64ba'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:12.138983 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-f59e92ce-fa0e-42f0-ac58-3774d58a64ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e338799da18942aa89350a3b4cfba6a9', 'request_id': 'req-f59e92ce-fa0e-42f0-ac58-3774d58a64ba'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:12.148968 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-f59e92ce-fa0e-42f0-ac58-3774d58a64ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e338799da18942aa89350a3b4cfba6a9', 'request_id': 'req-f59e92ce-fa0e-42f0-ac58-3774d58a64ba'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:12.177214 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-f59e92ce-fa0e-42f0-ac58-3774d58a64ba admin admin] The token cache is being invalidate because project e338799da18942aa89350a3b4cfba6a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 15:38:12.178127 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-f59e92ce-fa0e-42f0-ac58-3774d58a64ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:12.179126 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-f59e92ce-fa0e-42f0-ac58-3774d58a64ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:12.180959 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1041/2070] 10.222.0.99 () {58 vars in 1232 bytes} [Thu Jul 14 15:38:11 2022] DELETE /identity/v3/projects/e338799da18942aa89350a3b4cfba6a9 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 15:38:12.184927 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59e92ce-fa0e-42f0-ac58-3774d58a64ba admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:12.186582 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-f59e92ce-fa0e-42f0-ac58-3774d58a64ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:12.186806 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-3b2f7aba-107e-4010-88a5-5c160493e260 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70a38ee462734c6ab5c91aa3fbac669c'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:12.198609 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-3b2f7aba-107e-4010-88a5-5c160493e260 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70a38ee462734c6ab5c91aa3fbac669c'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:12.204945 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:12.225193 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-3b2f7aba-107e-4010-88a5-5c160493e260 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70a38ee462734c6ab5c91aa3fbac669c'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:12.226285 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-3b2f7aba-107e-4010-88a5-5c160493e260 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70a38ee462734c6ab5c91aa3fbac669c'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:12.227180 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-3b2f7aba-107e-4010-88a5-5c160493e260 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70a38ee462734c6ab5c91aa3fbac669c'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:12.246353 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1030/2071] 10.222.0.99 () {58 vars in 1223 bytes} [Thu Jul 14 15:38:11 2022] DELETE /identity/v3/users/70a38ee462734c6ab5c91aa3fbac669c => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 15:38:12.279324 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59e92ce-fa0e-42f0-ac58-3774d58a64ba admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:12.351202 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4e6bf4-4781-4fa0-9ba9-9abbbc093476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:12.352137 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4e6bf4-4781-4fa0-9ba9-9abbbc093476 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:12.352494 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4e6bf4-4781-4fa0-9ba9-9abbbc093476 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:12.352870 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4e6bf4-4781-4fa0-9ba9-9abbbc093476 admin admin] PATH_INFO: `/v3/users/e8b6ba6c235640338a3e1b526efcc632` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:12.354266 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4e6bf4-4781-4fa0-9ba9-9abbbc093476 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8b6ba6c235640338a3e1b526efcc632)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:12.356451 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4e6bf4-4781-4fa0-9ba9-9abbbc093476 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:12.427035 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-8f4e6bf4-4781-4fa0-9ba9-9abbbc093476 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8b6ba6c235640338a3e1b526efcc632'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:12.436761 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-8f4e6bf4-4781-4fa0-9ba9-9abbbc093476 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8b6ba6c235640338a3e1b526efcc632'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:12.455226 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2f7aba-107e-4010-88a5-5c160493e260 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:12.456901 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-3b2f7aba-107e-4010-88a5-5c160493e260 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:12.461568 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-8f4e6bf4-4781-4fa0-9ba9-9abbbc093476 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8b6ba6c235640338a3e1b526efcc632'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:12.462850 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-8f4e6bf4-4781-4fa0-9ba9-9abbbc093476 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8b6ba6c235640338a3e1b526efcc632'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:12.463883 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-8f4e6bf4-4781-4fa0-9ba9-9abbbc093476 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8b6ba6c235640338a3e1b526efcc632'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:12.474702 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2f7aba-107e-4010-88a5-5c160493e260 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:12.480275 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae5b503-e9b0-476d-8aae-2bfbd93f97bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:12.481164 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae5b503-e9b0-476d-8aae-2bfbd93f97bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:12.481596 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae5b503-e9b0-476d-8aae-2bfbd93f97bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:12.482025 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae5b503-e9b0-476d-8aae-2bfbd93f97bd admin admin] PATH_INFO: `/v3/projects/a597de03a82a4bb090c829e1beb8eaa7` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:12.483737 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae5b503-e9b0-476d-8aae-2bfbd93f97bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=a597de03a82a4bb090c829e1beb8eaa7)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:12.485790 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae5b503-e9b0-476d-8aae-2bfbd93f97bd admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:12.490261 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1042/2072] 10.222.0.99 () {58 vars in 1223 bytes} [Thu Jul 14 15:38:12 2022] DELETE /identity/v3/users/e8b6ba6c235640338a3e1b526efcc632 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 15:38:12.544448 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-aae5b503-e9b0-476d-8aae-2bfbd93f97bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a597de03a82a4bb090c829e1beb8eaa7', 'request_id': 'req-aae5b503-e9b0-476d-8aae-2bfbd93f97bd'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:12.551778 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-aae5b503-e9b0-476d-8aae-2bfbd93f97bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a597de03a82a4bb090c829e1beb8eaa7', 'request_id': 'req-aae5b503-e9b0-476d-8aae-2bfbd93f97bd'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:12.558308 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-aae5b503-e9b0-476d-8aae-2bfbd93f97bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a597de03a82a4bb090c829e1beb8eaa7', 'request_id': 'req-aae5b503-e9b0-476d-8aae-2bfbd93f97bd'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:12.575888 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-aae5b503-e9b0-476d-8aae-2bfbd93f97bd admin admin] The token cache is being invalidate because project a597de03a82a4bb090c829e1beb8eaa7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 15:38:12.576348 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-aae5b503-e9b0-476d-8aae-2bfbd93f97bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:12.577231 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-aae5b503-e9b0-476d-8aae-2bfbd93f97bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:12.580104 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1031/2073] 10.222.0.99 () {58 vars in 1232 bytes} [Thu Jul 14 15:38:12 2022] DELETE /identity/v3/projects/a597de03a82a4bb090c829e1beb8eaa7 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:38:12.585184 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290b21de-d722-485d-947b-30725fbb1882 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:12.585596 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290b21de-d722-485d-947b-30725fbb1882 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:12.585915 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290b21de-d722-485d-947b-30725fbb1882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:12.604817 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-290b21de-d722-485d-947b-30725fbb1882 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:38:12.606038 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-290b21de-d722-485d-947b-30725fbb1882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:12.650988 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:12.680074 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1043/2074] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:38:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:38:12.689272 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130e7cbb-bb79-4a46-bfe6-d8f9e1134c47 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:12.692950 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130e7cbb-bb79-4a46-bfe6-d8f9e1134c47 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:12.692950 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130e7cbb-bb79-4a46-bfe6-d8f9e1134c47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:12.711207 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-130e7cbb-bb79-4a46-bfe6-d8f9e1134c47 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:38:12.712491 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-130e7cbb-bb79-4a46-bfe6-d8f9e1134c47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:12.721043 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1032/2075] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:38:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 15:38:12.725326 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290b21de-d722-485d-947b-30725fbb1882 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:12.735625 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290b21de-d722-485d-947b-30725fbb1882 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:12.740547 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6645b09a-eaeb-4f7f-adb7-256f63cc2b77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:12.741219 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6645b09a-eaeb-4f7f-adb7-256f63cc2b77 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:12.741509 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6645b09a-eaeb-4f7f-adb7-256f63cc2b77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:12.741792 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6645b09a-eaeb-4f7f-adb7-256f63cc2b77 admin admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:12.742539 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6645b09a-eaeb-4f7f-adb7-256f63cc2b77 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:38:12.742851 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6645b09a-eaeb-4f7f-adb7-256f63cc2b77 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:12.744516 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6645b09a-eaeb-4f7f-adb7-256f63cc2b77 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:12.758523 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1044/2076] 10.222.0.99 () {58 vars in 1152 bytes} [Thu Jul 14 15:38:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:38:12.780504 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130e7cbb-bb79-4a46-bfe6-d8f9e1134c47 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:12.780649 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6645b09a-eaeb-4f7f-adb7-256f63cc2b77 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:12.782601 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-6645b09a-eaeb-4f7f-adb7-256f63cc2b77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:12.796416 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6645b09a-eaeb-4f7f-adb7-256f63cc2b77 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:12.801920 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e5994c-9f1b-4384-a4c5-1d8787b9c507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:12.802789 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e5994c-9f1b-4384-a4c5-1d8787b9c507 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:12.803072 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e5994c-9f1b-4384-a4c5-1d8787b9c507 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:12.803501 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e5994c-9f1b-4384-a4c5-1d8787b9c507 admin admin] PATH_INFO: `/v3/projects/3f76f2008b7c42b3a9dc99e9274ea78f` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:12.804764 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e5994c-9f1b-4384-a4c5-1d8787b9c507 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f76f2008b7c42b3a9dc99e9274ea78f)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:12.806691 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e5994c-9f1b-4384-a4c5-1d8787b9c507 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:12.816175 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130e7cbb-bb79-4a46-bfe6-d8f9e1134c47 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:12.820917 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bc628c-1bed-42a1-967c-2cf6118d3034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:12.821643 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bc628c-1bed-42a1-967c-2cf6118d3034 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:12.821909 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bc628c-1bed-42a1-967c-2cf6118d3034 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:12.822221 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bc628c-1bed-42a1-967c-2cf6118d3034 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:12.822911 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bc628c-1bed-42a1-967c-2cf6118d3034 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:12.824900 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bc628c-1bed-42a1-967c-2cf6118d3034 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:12.855719 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1033/2077] 10.222.0.99 () {60 vars in 1152 bytes} [Thu Jul 14 15:38:12 2022] POST /identity/v3/projects => generated 377 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:38:12.858885 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bc628c-1bed-42a1-967c-2cf6118d3034 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:12.877793 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-92e5994c-9f1b-4384-a4c5-1d8787b9c507 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f76f2008b7c42b3a9dc99e9274ea78f', 'request_id': 'req-92e5994c-9f1b-4384-a4c5-1d8787b9c507'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:12.888831 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-92e5994c-9f1b-4384-a4c5-1d8787b9c507 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f76f2008b7c42b3a9dc99e9274ea78f', 'request_id': 'req-92e5994c-9f1b-4384-a4c5-1d8787b9c507'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:12.898114 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-92e5994c-9f1b-4384-a4c5-1d8787b9c507 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f76f2008b7c42b3a9dc99e9274ea78f', 'request_id': 'req-92e5994c-9f1b-4384-a4c5-1d8787b9c507'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:12.913011 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:DELETE=17,keystone:SELECT=80,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:12.916903 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-92e5994c-9f1b-4384-a4c5-1d8787b9c507 admin admin] The token cache is being invalidate because project 3f76f2008b7c42b3a9dc99e9274ea78f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 15:38:12.917368 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-92e5994c-9f1b-4384-a4c5-1d8787b9c507 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:12.918200 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-92e5994c-9f1b-4384-a4c5-1d8787b9c507 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:12.919825 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1045/2078] 10.222.0.99 () {58 vars in 1232 bytes} [Thu Jul 14 15:38:12 2022] DELETE /identity/v3/projects/3f76f2008b7c42b3a9dc99e9274ea78f => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:38:12.928818 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5435b5-f633-4455-bb1f-3527502de90a None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:12.929166 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5435b5-f633-4455-bb1f-3527502de90a None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:12.929453 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5435b5-f633-4455-bb1f-3527502de90a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:12.942442 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bc628c-1bed-42a1-967c-2cf6118d3034 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:12.949229 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-3e5435b5-f633-4455-bb1f-3527502de90a None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:38:12.950466 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-3e5435b5-f633-4455-bb1f-3527502de90a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:13.014276 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56188a1d-3fac-439a-85a2-92e1e42a007b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:13.015118 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56188a1d-3fac-439a-85a2-92e1e42a007b admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:13.015519 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56188a1d-3fac-439a-85a2-92e1e42a007b admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:13.015970 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56188a1d-3fac-439a-85a2-92e1e42a007b admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:13.017570 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56188a1d-3fac-439a-85a2-92e1e42a007b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:13.020311 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56188a1d-3fac-439a-85a2-92e1e42a007b admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:13.034555 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1046/2079] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:38:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 15:38:13.042635 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc07edd-0c8c-4f38-8b97-e10a3387f0ec None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:13.042928 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc07edd-0c8c-4f38-8b97-e10a3387f0ec None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:13.043582 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc07edd-0c8c-4f38-8b97-e10a3387f0ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:13.049421 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1034/2080] 10.222.0.99 () {60 vars in 1143 bytes} [Thu Jul 14 15:38:12 2022] POST /identity/v3/users => generated 289 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:38:13.050469 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56188a1d-3fac-439a-85a2-92e1e42a007b admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:13.052343 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-56188a1d-3fac-439a-85a2-92e1e42a007b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:13.064754 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-5bc07edd-0c8c-4f38-8b97-e10a3387f0ec None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:38:13.064987 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56188a1d-3fac-439a-85a2-92e1e42a007b admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:13.065965 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-5bc07edd-0c8c-4f38-8b97-e10a3387f0ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:13.070075 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609d7cf0-310e-445f-9d80-861e49c2f362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:13.070884 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609d7cf0-310e-445f-9d80-861e49c2f362 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:13.071254 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609d7cf0-310e-445f-9d80-861e49c2f362 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:13.071712 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609d7cf0-310e-445f-9d80-861e49c2f362 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:13.072452 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609d7cf0-310e-445f-9d80-861e49c2f362 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:13.074155 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609d7cf0-310e-445f-9d80-861e49c2f362 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:13.079115 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1047/2081] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:38:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 15:38:13.080346 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc07edd-0c8c-4f38-8b97-e10a3387f0ec None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:13.090139 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1035/2082] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:38:13 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:38:13.090381 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc07edd-0c8c-4f38-8b97-e10a3387f0ec None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:13.094186 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609d7cf0-310e-445f-9d80-861e49c2f362 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:13.096232 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac71e92-e8da-491d-b28d-02875ebe9787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:13.096899 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac71e92-e8da-491d-b28d-02875ebe9787 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:13.097184 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac71e92-e8da-491d-b28d-02875ebe9787 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:13.097459 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac71e92-e8da-491d-b28d-02875ebe9787 admin admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:13.098192 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac71e92-e8da-491d-b28d-02875ebe9787 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:38:13.098496 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac71e92-e8da-491d-b28d-02875ebe9787 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:13.100190 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac71e92-e8da-491d-b28d-02875ebe9787 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:13.106724 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609d7cf0-310e-445f-9d80-861e49c2f362 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:13.112830 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1048/2083] 10.222.0.99 () {58 vars in 1152 bytes} [Thu Jul 14 15:38:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:38:13.115583 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea31cec3-3286-446a-b783-9b75ab864da6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:13.116578 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea31cec3-3286-446a-b783-9b75ab864da6 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:13.117029 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea31cec3-3286-446a-b783-9b75ab864da6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:13.117375 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea31cec3-3286-446a-b783-9b75ab864da6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:13.117488 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac71e92-e8da-491d-b28d-02875ebe9787 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:13.118457 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea31cec3-3286-446a-b783-9b75ab864da6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:13.120213 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea31cec3-3286-446a-b783-9b75ab864da6 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:13.128806 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac71e92-e8da-491d-b28d-02875ebe9787 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:13.135366 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1036/2084] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:38:13 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:38:13.138988 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62337a7-46d4-46dc-b78d-a5d04178a002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:13.139854 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62337a7-46d4-46dc-b78d-a5d04178a002 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:13.140105 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea31cec3-3286-446a-b783-9b75ab864da6 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:13.141129 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62337a7-46d4-46dc-b78d-a5d04178a002 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:13.141129 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62337a7-46d4-46dc-b78d-a5d04178a002 admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:13.141663 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62337a7-46d4-46dc-b78d-a5d04178a002 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:13.143938 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62337a7-46d4-46dc-b78d-a5d04178a002 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:13.156437 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea31cec3-3286-446a-b783-9b75ab864da6 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:13.166259 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0659a8-cd6c-42e9-b0f9-7efb6ea29aa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:13.167245 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0659a8-cd6c-42e9-b0f9-7efb6ea29aa3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:13.167844 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0659a8-cd6c-42e9-b0f9-7efb6ea29aa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:13.168295 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0659a8-cd6c-42e9-b0f9-7efb6ea29aa3 admin admin] PATH_INFO: `/v3/projects/df766904ce8c45fc9eb0b59e959f8583/users/e34e7260a8e8454ab282db7e8a50169e/roles/b8c98fddc0fd4436b577fd1d3361bc38` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:13.190360 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1049/2085] 10.222.0.99 () {60 vars in 1152 bytes} [Thu Jul 14 15:38:13 2022] POST /identity/v3/projects => generated 375 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 15:38:13.195037 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62337a7-46d4-46dc-b78d-a5d04178a002 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:13.195673 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0659a8-cd6c-42e9-b0f9-7efb6ea29aa3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df766904ce8c45fc9eb0b59e959f8583, user_id=e34e7260a8e8454ab282db7e8a50169e, role_id=b8c98fddc0fd4436b577fd1d3361bc38)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:13.198569 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0659a8-cd6c-42e9-b0f9-7efb6ea29aa3 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:13.212544 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:13.218546 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1037/2086] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 15:38:13 2022] PUT /identity/v3/projects/df766904ce8c45fc9eb0b59e959f8583/users/e34e7260a8e8454ab282db7e8a50169e/roles/b8c98fddc0fd4436b577fd1d3361bc38 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:38:13.229336 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6c5b4c-c4cb-4ec0-bebb-5f6c1c9b489f None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:13.229750 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6c5b4c-c4cb-4ec0-bebb-5f6c1c9b489f None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:13.232543 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6c5b4c-c4cb-4ec0-bebb-5f6c1c9b489f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:13.274153 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-8e6c5b4c-c4cb-4ec0-bebb-5f6c1c9b489f None None] MFA Rules not processed for user `e34e7260a8e8454ab282db7e8a50169e`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:38:13.275383 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-8e6c5b4c-c4cb-4ec0-bebb-5f6c1c9b489f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:13.283237 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62337a7-46d4-46dc-b78d-a5d04178a002 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:13.352168 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1038/2087] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:38:13 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 15:38:13.354840 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b20787-5be9-4e8a-acb0-6b236af857ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:13.355536 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b20787-5be9-4e8a-acb0-6b236af857ab admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:13.355858 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b20787-5be9-4e8a-acb0-6b236af857ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:13.356140 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b20787-5be9-4e8a-acb0-6b236af857ab admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:13.356911 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b20787-5be9-4e8a-acb0-6b236af857ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:13.359030 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b20787-5be9-4e8a-acb0-6b236af857ab admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:13.365030 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6c5b4c-c4cb-4ec0-bebb-5f6c1c9b489f None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:13.366822 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-8e6c5b4c-c4cb-4ec0-bebb-5f6c1c9b489f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:13.381933 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1050/2088] 10.222.0.99 () {60 vars in 1143 bytes} [Thu Jul 14 15:38:13 2022] POST /identity/v3/users => generated 288 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:38:13.386292 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b20787-5be9-4e8a-acb0-6b236af857ab admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:13.397660 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b20787-5be9-4e8a-acb0-6b236af857ab admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:13.403144 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44144ecd-2e49-4fc4-b932-84909ac8c8e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:13.404032 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44144ecd-2e49-4fc4-b932-84909ac8c8e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:13.404420 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44144ecd-2e49-4fc4-b932-84909ac8c8e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:13.404799 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44144ecd-2e49-4fc4-b932-84909ac8c8e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:13.405405 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:13.405725 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44144ecd-2e49-4fc4-b932-84909ac8c8e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:13.408005 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44144ecd-2e49-4fc4-b932-84909ac8c8e3 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:13.419747 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1051/2089] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:38:13 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:38:13.424781 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44144ecd-2e49-4fc4-b932-84909ac8c8e3 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:13.434478 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44144ecd-2e49-4fc4-b932-84909ac8c8e3 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:13.439574 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6430d9c-dac5-4d9d-b744-f24067575b46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:13.440324 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6430d9c-dac5-4d9d-b744-f24067575b46 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:13.440616 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6430d9c-dac5-4d9d-b744-f24067575b46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:13.440893 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6430d9c-dac5-4d9d-b744-f24067575b46 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:13.441620 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6430d9c-dac5-4d9d-b744-f24067575b46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:13.443217 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6430d9c-dac5-4d9d-b744-f24067575b46 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:13.443808 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6c5b4c-c4cb-4ec0-bebb-5f6c1c9b489f None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:13.447742 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a62a9f-d1aa-40c0-bf2c-4f6c80aba318 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3e1a2b83f3d4d88ba5084dfaffd22f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:13.448454 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a62a9f-d1aa-40c0-bf2c-4f6c80aba318 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:13.448750 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a62a9f-d1aa-40c0-bf2c-4f6c80aba318 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:13.449063 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a62a9f-d1aa-40c0-bf2c-4f6c80aba318 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:13.455135 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1052/2090] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:38:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 15:38:13.459114 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6430d9c-dac5-4d9d-b744-f24067575b46 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:13.461064 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a62a9f-d1aa-40c0-bf2c-4f6c80aba318 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:13.462520 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a62a9f-d1aa-40c0-bf2c-4f6c80aba318 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:13.470723 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6430d9c-dac5-4d9d-b744-f24067575b46 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:13.472896 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1039/2091] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:38:13 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 15:38:13.475486 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0460bb73-2c44-4a08-b045-d7c825c35745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:13.476402 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0460bb73-2c44-4a08-b045-d7c825c35745 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:13.476693 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0460bb73-2c44-4a08-b045-d7c825c35745 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:13.476979 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0460bb73-2c44-4a08-b045-d7c825c35745 admin admin] PATH_INFO: `/v3/projects/22afea9c5f524cb8b7de7790eeb4a220/users/d1456f1f29f54585882a2a4872287e8a/roles/b8c98fddc0fd4436b577fd1d3361bc38` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:13.492819 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0460bb73-2c44-4a08-b045-d7c825c35745 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22afea9c5f524cb8b7de7790eeb4a220, user_id=d1456f1f29f54585882a2a4872287e8a, role_id=b8c98fddc0fd4436b577fd1d3361bc38)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:13.495489 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0460bb73-2c44-4a08-b045-d7c825c35745 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:13.509847 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1053/2092] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 15:38:13 2022] PUT /identity/v3/projects/22afea9c5f524cb8b7de7790eeb4a220/users/d1456f1f29f54585882a2a4872287e8a/roles/b8c98fddc0fd4436b577fd1d3361bc38 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 15:38:13.520871 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6f2bc4-845e-4d23-a92e-96b0869c18eb None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:13.521091 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6f2bc4-845e-4d23-a92e-96b0869c18eb None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:13.521678 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6f2bc4-845e-4d23-a92e-96b0869c18eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:13.558477 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-4b6f2bc4-845e-4d23-a92e-96b0869c18eb None None] MFA Rules not processed for user `d1456f1f29f54585882a2a4872287e8a`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:38:13.559740 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-4b6f2bc4-845e-4d23-a92e-96b0869c18eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:13.629258 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1040/2093] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:38:13 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:38:13.642121 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0460bb73-2c44-4a08-b045-d7c825c35745 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:13.644087 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-0460bb73-2c44-4a08-b045-d7c825c35745 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:13.715089 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0460bb73-2c44-4a08-b045-d7c825c35745 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:13.719976 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4515cb-fa71-496a-8897-9d8fbf826442 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3e1a2b83f3d4d88ba5084dfaffd22f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:13.720243 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4515cb-fa71-496a-8897-9d8fbf826442 service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:13.720532 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4515cb-fa71-496a-8897-9d8fbf826442 service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:13.720813 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4515cb-fa71-496a-8897-9d8fbf826442 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:13.730936 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4515cb-fa71-496a-8897-9d8fbf826442 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:13.732584 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4515cb-fa71-496a-8897-9d8fbf826442 service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:13.747815 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1054/2094] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:38:13 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 15:38:19.033601 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6f2bc4-845e-4d23-a92e-96b0869c18eb None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:19.035225 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-4b6f2bc4-845e-4d23-a92e-96b0869c18eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:19.104394 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6f2bc4-845e-4d23-a92e-96b0869c18eb None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:19.108892 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678221d9-5bb9-40d7-9d75-39ef88f8d6d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:19.109755 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678221d9-5bb9-40d7-9d75-39ef88f8d6d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:19.110036 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678221d9-5bb9-40d7-9d75-39ef88f8d6d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:19.110318 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678221d9-5bb9-40d7-9d75-39ef88f8d6d8 admin admin] PATH_INFO: `/v3/users/3db7ba60a24747efb55601b6e2401983` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:19.112069 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678221d9-5bb9-40d7-9d75-39ef88f8d6d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3db7ba60a24747efb55601b6e2401983)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:19.113994 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678221d9-5bb9-40d7-9d75-39ef88f8d6d8 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:19.178642 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-678221d9-5bb9-40d7-9d75-39ef88f8d6d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3db7ba60a24747efb55601b6e2401983'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:19.186279 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-678221d9-5bb9-40d7-9d75-39ef88f8d6d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3db7ba60a24747efb55601b6e2401983'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:19.188237 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:19.202294 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-678221d9-5bb9-40d7-9d75-39ef88f8d6d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3db7ba60a24747efb55601b6e2401983'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:19.203212 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-678221d9-5bb9-40d7-9d75-39ef88f8d6d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3db7ba60a24747efb55601b6e2401983'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:19.203860 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-678221d9-5bb9-40d7-9d75-39ef88f8d6d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3db7ba60a24747efb55601b6e2401983'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:19.216663 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1041/2095] 10.222.0.99 () {58 vars in 1223 bytes} [Thu Jul 14 15:38:19 2022] DELETE /identity/v3/users/3db7ba60a24747efb55601b6e2401983 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:38:19.402415 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4515cb-fa71-496a-8897-9d8fbf826442 service cinder] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:19.409147 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:19.418741 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4515cb-fa71-496a-8897-9d8fbf826442 service cinder] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:19.494273 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca0d6cf-3001-49c5-96ee-add404c429c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:19.495151 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca0d6cf-3001-49c5-96ee-add404c429c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:19.495464 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca0d6cf-3001-49c5-96ee-add404c429c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:19.495779 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca0d6cf-3001-49c5-96ee-add404c429c0 admin admin] PATH_INFO: `/v3/projects/b9490b542a1f4c78aef9015ecb97dabd` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:19.497102 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca0d6cf-3001-49c5-96ee-add404c429c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9490b542a1f4c78aef9015ecb97dabd)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:19.499075 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca0d6cf-3001-49c5-96ee-add404c429c0 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:19.549776 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-7ca0d6cf-3001-49c5-96ee-add404c429c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9490b542a1f4c78aef9015ecb97dabd', 'request_id': 'req-7ca0d6cf-3001-49c5-96ee-add404c429c0'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:19.556113 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-7ca0d6cf-3001-49c5-96ee-add404c429c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9490b542a1f4c78aef9015ecb97dabd', 'request_id': 'req-7ca0d6cf-3001-49c5-96ee-add404c429c0'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:19.562190 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-7ca0d6cf-3001-49c5-96ee-add404c429c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9490b542a1f4c78aef9015ecb97dabd', 'request_id': 'req-7ca0d6cf-3001-49c5-96ee-add404c429c0'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:19.574667 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-7ca0d6cf-3001-49c5-96ee-add404c429c0 admin admin] The token cache is being invalidate because project b9490b542a1f4c78aef9015ecb97dabd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 15:38:19.574959 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-7ca0d6cf-3001-49c5-96ee-add404c429c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:19.575588 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-7ca0d6cf-3001-49c5-96ee-add404c429c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:19.577155 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1055/2096] 10.222.0.99 () {58 vars in 1232 bytes} [Thu Jul 14 15:38:19 2022] DELETE /identity/v3/projects/b9490b542a1f4c78aef9015ecb97dabd => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:38:19.582788 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8656dd46-4660-4dd8-b985-802765d52908 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:19.583155 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8656dd46-4660-4dd8-b985-802765d52908 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:19.583484 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8656dd46-4660-4dd8-b985-802765d52908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:19.603854 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-8656dd46-4660-4dd8-b985-802765d52908 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:38:19.605080 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-8656dd46-4660-4dd8-b985-802765d52908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:19.665336 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1042/2097] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:38:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:38:19.674152 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5331e5a-4d08-4c8f-bef6-0650c10a8f07 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:19.674451 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5331e5a-4d08-4c8f-bef6-0650c10a8f07 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:19.674740 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5331e5a-4d08-4c8f-bef6-0650c10a8f07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:19.693435 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-a5331e5a-4d08-4c8f-bef6-0650c10a8f07 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:38:19.694663 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-a5331e5a-4d08-4c8f-bef6-0650c10a8f07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:19.702714 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1056/2098] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:38:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:38:19.707528 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8656dd46-4660-4dd8-b985-802765d52908 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:19.717547 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8656dd46-4660-4dd8-b985-802765d52908 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:19.723170 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec428a74-cc3d-4399-bbce-771a278b59f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:19.723872 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec428a74-cc3d-4399-bbce-771a278b59f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:19.724166 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec428a74-cc3d-4399-bbce-771a278b59f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:19.724451 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec428a74-cc3d-4399-bbce-771a278b59f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:19.725193 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec428a74-cc3d-4399-bbce-771a278b59f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:38:19.725546 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec428a74-cc3d-4399-bbce-771a278b59f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:19.727159 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec428a74-cc3d-4399-bbce-771a278b59f1 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:19.739647 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1043/2099] 10.222.0.99 () {58 vars in 1152 bytes} [Thu Jul 14 15:38:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:38:19.744240 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5331e5a-4d08-4c8f-bef6-0650c10a8f07 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:19.755115 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5331e5a-4d08-4c8f-bef6-0650c10a8f07 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:19.759802 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5ea6a9-bfd7-4382-9d45-263e06aa9184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:19.760495 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5ea6a9-bfd7-4382-9d45-263e06aa9184 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:19.760788 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5ea6a9-bfd7-4382-9d45-263e06aa9184 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:19.761070 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5ea6a9-bfd7-4382-9d45-263e06aa9184 admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:19.761777 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5ea6a9-bfd7-4382-9d45-263e06aa9184 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:19.763711 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5ea6a9-bfd7-4382-9d45-263e06aa9184 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:19.795637 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1057/2100] 10.222.0.99 () {60 vars in 1152 bytes} [Thu Jul 14 15:38:19 2022] POST /identity/v3/projects => generated 365 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:38:19.797764 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec428a74-cc3d-4399-bbce-771a278b59f1 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:19.808033 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec428a74-cc3d-4399-bbce-771a278b59f1 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:19.867060 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add4107e-bb20-4f19-ab42-4efdecab13fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:19.867806 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add4107e-bb20-4f19-ab42-4efdecab13fe admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:19.868104 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add4107e-bb20-4f19-ab42-4efdecab13fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:19.868391 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add4107e-bb20-4f19-ab42-4efdecab13fe admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:19.869102 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add4107e-bb20-4f19-ab42-4efdecab13fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:19.871033 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add4107e-bb20-4f19-ab42-4efdecab13fe admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:19.892784 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1044/2101] 10.222.0.99 () {60 vars in 1143 bytes} [Thu Jul 14 15:38:19 2022] POST /identity/v3/users => generated 283 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 15:38:19.897621 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5ea6a9-bfd7-4382-9d45-263e06aa9184 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:19.909847 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5ea6a9-bfd7-4382-9d45-263e06aa9184 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:19.915773 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dc3cf6-ede6-4c43-9c37-62e5b81d23d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:19.916422 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dc3cf6-ede6-4c43-9c37-62e5b81d23d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:19.916705 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dc3cf6-ede6-4c43-9c37-62e5b81d23d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:19.917018 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dc3cf6-ede6-4c43-9c37-62e5b81d23d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:19.917751 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dc3cf6-ede6-4c43-9c37-62e5b81d23d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:19.919920 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dc3cf6-ede6-4c43-9c37-62e5b81d23d7 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:19.932526 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1058/2102] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:38:19 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 15:38:19.936627 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add4107e-bb20-4f19-ab42-4efdecab13fe admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:19.948294 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add4107e-bb20-4f19-ab42-4efdecab13fe admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:19.953158 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782eb3a5-df7e-4578-88bf-a50f773708ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:19.953854 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782eb3a5-df7e-4578-88bf-a50f773708ac admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:19.954144 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782eb3a5-df7e-4578-88bf-a50f773708ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:19.954447 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782eb3a5-df7e-4578-88bf-a50f773708ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:19.956343 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782eb3a5-df7e-4578-88bf-a50f773708ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:19.958299 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782eb3a5-df7e-4578-88bf-a50f773708ac admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:19.968715 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1045/2103] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:38:19 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:38:19.973395 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dc3cf6-ede6-4c43-9c37-62e5b81d23d7 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:19.984012 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dc3cf6-ede6-4c43-9c37-62e5b81d23d7 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:19.989296 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6194e9-3a1f-4032-95ab-1a86e8353be3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:19.991221 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6194e9-3a1f-4032-95ab-1a86e8353be3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:19.991221 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6194e9-3a1f-4032-95ab-1a86e8353be3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:19.991221 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6194e9-3a1f-4032-95ab-1a86e8353be3 admin admin] PATH_INFO: `/v3/projects/d4a13f1a305f48feab1d50bde7189121/users/616d8f36f9b04691a859d5b73a58188b/roles/b8c98fddc0fd4436b577fd1d3361bc38` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:20.008311 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6194e9-3a1f-4032-95ab-1a86e8353be3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4a13f1a305f48feab1d50bde7189121, user_id=616d8f36f9b04691a859d5b73a58188b, role_id=b8c98fddc0fd4436b577fd1d3361bc38)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:20.011045 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6194e9-3a1f-4032-95ab-1a86e8353be3 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:20.026998 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1059/2104] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 15:38:19 2022] PUT /identity/v3/projects/d4a13f1a305f48feab1d50bde7189121/users/616d8f36f9b04691a859d5b73a58188b/roles/b8c98fddc0fd4436b577fd1d3361bc38 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:38:20.037742 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a03eb4-ff82-4e6b-a29a-a0acfdbf5383 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:20.038236 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a03eb4-ff82-4e6b-a29a-a0acfdbf5383 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:20.038353 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a03eb4-ff82-4e6b-a29a-a0acfdbf5383 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:20.068686 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-98a03eb4-ff82-4e6b-a29a-a0acfdbf5383 None None] MFA Rules not processed for user `616d8f36f9b04691a859d5b73a58188b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:38:20.070124 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-98a03eb4-ff82-4e6b-a29a-a0acfdbf5383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:20.104852 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:20.144292 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1046/2105] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:38:20 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:38:20.149189 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6194e9-3a1f-4032-95ab-1a86e8353be3 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:20.181995 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:20.222989 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6194e9-3a1f-4032-95ab-1a86e8353be3 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:20.227718 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f126784a-8ffe-4c84-ba08-ce0e34aa97f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:20.228395 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f126784a-8ffe-4c84-ba08-ce0e34aa97f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:20.228683 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f126784a-8ffe-4c84-ba08-ce0e34aa97f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:20.228992 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f126784a-8ffe-4c84-ba08-ce0e34aa97f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:20.229713 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f126784a-8ffe-4c84-ba08-ce0e34aa97f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:20.231638 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f126784a-8ffe-4c84-ba08-ce0e34aa97f5 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:20.262521 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1060/2106] 10.222.0.99 () {60 vars in 1152 bytes} [Thu Jul 14 15:38:20 2022] POST /identity/v3/projects => generated 365 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:38:20.266726 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a03eb4-ff82-4e6b-a29a-a0acfdbf5383 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:20.279213 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a03eb4-ff82-4e6b-a29a-a0acfdbf5383 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:20.349706 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da4fbf1-dc60-4446-8dfa-c9716b348525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:20.349706 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da4fbf1-dc60-4446-8dfa-c9716b348525 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:20.349706 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da4fbf1-dc60-4446-8dfa-c9716b348525 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:20.349706 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da4fbf1-dc60-4446-8dfa-c9716b348525 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:20.350300 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da4fbf1-dc60-4446-8dfa-c9716b348525 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:20.351880 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da4fbf1-dc60-4446-8dfa-c9716b348525 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:20.374707 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1047/2107] 10.222.0.99 () {60 vars in 1143 bytes} [Thu Jul 14 15:38:20 2022] POST /identity/v3/users => generated 289 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:38:20.378802 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f126784a-8ffe-4c84-ba08-ce0e34aa97f5 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:20.390082 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f126784a-8ffe-4c84-ba08-ce0e34aa97f5 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:20.395822 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcac6d6e-097f-4774-9275-19053b296a8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:20.396759 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcac6d6e-097f-4774-9275-19053b296a8f admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:20.397046 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcac6d6e-097f-4774-9275-19053b296a8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:20.397325 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcac6d6e-097f-4774-9275-19053b296a8f admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:20.399448 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcac6d6e-097f-4774-9275-19053b296a8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:20.399707 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcac6d6e-097f-4774-9275-19053b296a8f admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:20.411067 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1061/2108] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:38:20 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:38:20.415304 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da4fbf1-dc60-4446-8dfa-c9716b348525 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:20.426530 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da4fbf1-dc60-4446-8dfa-c9716b348525 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:20.431035 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d70af2-5657-4a5f-8b0d-6b89f43fd98c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:20.431944 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d70af2-5657-4a5f-8b0d-6b89f43fd98c admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:20.432234 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d70af2-5657-4a5f-8b0d-6b89f43fd98c admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:20.432516 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d70af2-5657-4a5f-8b0d-6b89f43fd98c admin admin] PATH_INFO: `/v3/projects/1b7c6506581244188949a54eddd6f6aa/users/cc2e8ac4c2094bd8ba8546346a3f065e/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:20.449784 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d70af2-5657-4a5f-8b0d-6b89f43fd98c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b7c6506581244188949a54eddd6f6aa, user_id=cc2e8ac4c2094bd8ba8546346a3f065e, role_id=d0f6ba1314f14d9f9c816bdb5d0c3aac)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:20.452490 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d70af2-5657-4a5f-8b0d-6b89f43fd98c admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:20.468487 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1048/2109] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 15:38:20 2022] PUT /identity/v3/projects/1b7c6506581244188949a54eddd6f6aa/users/cc2e8ac4c2094bd8ba8546346a3f065e/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:38:20.473541 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcac6d6e-097f-4774-9275-19053b296a8f admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:20.483969 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcac6d6e-097f-4774-9275-19053b296a8f admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:20.542741 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65df3bd-e20d-44e1-975b-36d9d20d3898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:20.543421 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65df3bd-e20d-44e1-975b-36d9d20d3898 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:20.543740 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65df3bd-e20d-44e1-975b-36d9d20d3898 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:20.544019 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65df3bd-e20d-44e1-975b-36d9d20d3898 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:20.544757 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65df3bd-e20d-44e1-975b-36d9d20d3898 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:20.546377 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65df3bd-e20d-44e1-975b-36d9d20d3898 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:20.557292 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1062/2110] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:38:20 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:38:20.561547 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d70af2-5657-4a5f-8b0d-6b89f43fd98c admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:20.573579 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d70af2-5657-4a5f-8b0d-6b89f43fd98c admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:20.578354 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b314e86-0beb-440a-880b-968f763af0f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:20.579374 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b314e86-0beb-440a-880b-968f763af0f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:20.579909 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b314e86-0beb-440a-880b-968f763af0f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:20.580204 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b314e86-0beb-440a-880b-968f763af0f2 admin admin] PATH_INFO: `/v3/projects/1b7c6506581244188949a54eddd6f6aa/users/cc2e8ac4c2094bd8ba8546346a3f065e/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:20.582385 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b314e86-0beb-440a-880b-968f763af0f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b7c6506581244188949a54eddd6f6aa, user_id=cc2e8ac4c2094bd8ba8546346a3f065e, role_id=d0f6ba1314f14d9f9c816bdb5d0c3aac)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:20.584884 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b314e86-0beb-440a-880b-968f763af0f2 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:20.596050 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1049/2111] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 15:38:20 2022] PUT /identity/v3/projects/1b7c6506581244188949a54eddd6f6aa/users/cc2e8ac4c2094bd8ba8546346a3f065e/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:38:20.606117 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c301e3-74e6-40ec-9a34-a449fe17eb27 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:20.606415 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c301e3-74e6-40ec-9a34-a449fe17eb27 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:20.606703 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c301e3-74e6-40ec-9a34-a449fe17eb27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:20.634780 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-33c301e3-74e6-40ec-9a34-a449fe17eb27 None None] MFA Rules not processed for user `cc2e8ac4c2094bd8ba8546346a3f065e`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:38:20.636053 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-33c301e3-74e6-40ec-9a34-a449fe17eb27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:20.708275 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1063/2112] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:38:20 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:38:20.721397 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b314e86-0beb-440a-880b-968f763af0f2 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:20.723141 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-8b314e86-0beb-440a-880b-968f763af0f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:20.789506 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b314e86-0beb-440a-880b-968f763af0f2 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:20.793145 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30205677-3c5a-4c80-9dff-94c571d846c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3e1a2b83f3d4d88ba5084dfaffd22f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:20.793794 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30205677-3c5a-4c80-9dff-94c571d846c9 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:20.794081 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30205677-3c5a-4c80-9dff-94c571d846c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:20.794393 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30205677-3c5a-4c80-9dff-94c571d846c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:20.805268 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30205677-3c5a-4c80-9dff-94c571d846c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:20.806710 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30205677-3c5a-4c80-9dff-94c571d846c9 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:20.817556 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1050/2113] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:38:20 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:38:21.259790 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c301e3-74e6-40ec-9a34-a449fe17eb27 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:21.289057 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:21.326494 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c301e3-74e6-40ec-9a34-a449fe17eb27 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:21.331462 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6ed6fa-a7a3-416c-bc76-3bab25f5a2dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:21.332371 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6ed6fa-a7a3-416c-bc76-3bab25f5a2dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:21.332661 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6ed6fa-a7a3-416c-bc76-3bab25f5a2dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:21.332941 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6ed6fa-a7a3-416c-bc76-3bab25f5a2dd admin admin] PATH_INFO: `/v3/users/616d8f36f9b04691a859d5b73a58188b` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:21.334133 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6ed6fa-a7a3-416c-bc76-3bab25f5a2dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=616d8f36f9b04691a859d5b73a58188b)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:21.336108 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6ed6fa-a7a3-416c-bc76-3bab25f5a2dd admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:21.383080 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-cf6ed6fa-a7a3-416c-bc76-3bab25f5a2dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '616d8f36f9b04691a859d5b73a58188b'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:21.391445 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-cf6ed6fa-a7a3-416c-bc76-3bab25f5a2dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '616d8f36f9b04691a859d5b73a58188b'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:21.404228 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-cf6ed6fa-a7a3-416c-bc76-3bab25f5a2dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '616d8f36f9b04691a859d5b73a58188b'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:21.404922 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-cf6ed6fa-a7a3-416c-bc76-3bab25f5a2dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '616d8f36f9b04691a859d5b73a58188b'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:21.405679 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-cf6ed6fa-a7a3-416c-bc76-3bab25f5a2dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '616d8f36f9b04691a859d5b73a58188b'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:21.418990 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1064/2114] 10.222.0.99 () {58 vars in 1223 bytes} [Thu Jul 14 15:38:21 2022] DELETE /identity/v3/users/616d8f36f9b04691a859d5b73a58188b => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:38:21.424370 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61c4f0d-8c6a-4ea9-b561-a34a227acb0d None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:21.424985 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61c4f0d-8c6a-4ea9-b561-a34a227acb0d None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:21.424985 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61c4f0d-8c6a-4ea9-b561-a34a227acb0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:21.442172 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-b61c4f0d-8c6a-4ea9-b561-a34a227acb0d None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:38:21.443376 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b61c4f0d-8c6a-4ea9-b561-a34a227acb0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:21.461181 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:21.504394 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1051/2115] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:38:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:38:21.513901 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6ed6fa-a7a3-416c-bc76-3bab25f5a2dd admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:21.515347 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-cf6ed6fa-a7a3-416c-bc76-3bab25f5a2dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:21.577469 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6ed6fa-a7a3-416c-bc76-3bab25f5a2dd admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:21.581373 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9166bf15-a16d-4a23-b275-4985e72e6c65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:21.582039 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9166bf15-a16d-4a23-b275-4985e72e6c65 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:21.582330 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9166bf15-a16d-4a23-b275-4985e72e6c65 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:21.582610 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9166bf15-a16d-4a23-b275-4985e72e6c65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:21.592084 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9166bf15-a16d-4a23-b275-4985e72e6c65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:21.593580 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9166bf15-a16d-4a23-b275-4985e72e6c65 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:21.603813 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1065/2116] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:38:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:38:21.950514 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61c4f0d-8c6a-4ea9-b561-a34a227acb0d None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:21.952013 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b61c4f0d-8c6a-4ea9-b561-a34a227acb0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:21.965816 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61c4f0d-8c6a-4ea9-b561-a34a227acb0d None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:21.970769 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5950ef-cf4f-4700-953e-672e4414c30b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:21.971660 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5950ef-cf4f-4700-953e-672e4414c30b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:21.972137 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5950ef-cf4f-4700-953e-672e4414c30b admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:21.972402 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5950ef-cf4f-4700-953e-672e4414c30b admin admin] PATH_INFO: `/v3/projects/d4a13f1a305f48feab1d50bde7189121` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:21.973850 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5950ef-cf4f-4700-953e-672e4414c30b admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4a13f1a305f48feab1d50bde7189121)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:21.975882 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5950ef-cf4f-4700-953e-672e4414c30b admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:22.032984 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2b5950ef-cf4f-4700-953e-672e4414c30b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4a13f1a305f48feab1d50bde7189121', 'request_id': 'req-2b5950ef-cf4f-4700-953e-672e4414c30b'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:22.039216 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2b5950ef-cf4f-4700-953e-672e4414c30b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4a13f1a305f48feab1d50bde7189121', 'request_id': 'req-2b5950ef-cf4f-4700-953e-672e4414c30b'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:22.044853 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2b5950ef-cf4f-4700-953e-672e4414c30b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4a13f1a305f48feab1d50bde7189121', 'request_id': 'req-2b5950ef-cf4f-4700-953e-672e4414c30b'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:22.059040 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2b5950ef-cf4f-4700-953e-672e4414c30b admin admin] The token cache is being invalidate because project d4a13f1a305f48feab1d50bde7189121 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 15:38:22.059336 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2b5950ef-cf4f-4700-953e-672e4414c30b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:22.059988 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2b5950ef-cf4f-4700-953e-672e4414c30b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:22.061747 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1052/2117] 10.222.0.99 () {58 vars in 1232 bytes} [Thu Jul 14 15:38:21 2022] DELETE /identity/v3/projects/d4a13f1a305f48feab1d50bde7189121 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:38:22.066568 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9166bf15-a16d-4a23-b275-4985e72e6c65 service neutron] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:22.076717 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9166bf15-a16d-4a23-b275-4985e72e6c65 service neutron] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:22.122788 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:22.138396 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee20fd5-3d65-4b47-a65a-72a0edeed4ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:22.139278 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee20fd5-3d65-4b47-a65a-72a0edeed4ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:22.139572 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee20fd5-3d65-4b47-a65a-72a0edeed4ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:22.139887 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee20fd5-3d65-4b47-a65a-72a0edeed4ee admin admin] PATH_INFO: `/v3/users/cc2e8ac4c2094bd8ba8546346a3f065e` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:22.141086 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee20fd5-3d65-4b47-a65a-72a0edeed4ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc2e8ac4c2094bd8ba8546346a3f065e)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:22.143379 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee20fd5-3d65-4b47-a65a-72a0edeed4ee admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:22.193252 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-0ee20fd5-3d65-4b47-a65a-72a0edeed4ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc2e8ac4c2094bd8ba8546346a3f065e'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:22.200874 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-0ee20fd5-3d65-4b47-a65a-72a0edeed4ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc2e8ac4c2094bd8ba8546346a3f065e'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:22.214599 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-0ee20fd5-3d65-4b47-a65a-72a0edeed4ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc2e8ac4c2094bd8ba8546346a3f065e'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:22.215286 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-0ee20fd5-3d65-4b47-a65a-72a0edeed4ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc2e8ac4c2094bd8ba8546346a3f065e'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:22.215911 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-0ee20fd5-3d65-4b47-a65a-72a0edeed4ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc2e8ac4c2094bd8ba8546346a3f065e'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:22.229659 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1066/2118] 10.222.0.99 () {58 vars in 1223 bytes} [Thu Jul 14 15:38:22 2022] DELETE /identity/v3/users/cc2e8ac4c2094bd8ba8546346a3f065e => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:38:22.512763 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5950ef-cf4f-4700-953e-672e4414c30b admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:22.514341 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-2b5950ef-cf4f-4700-953e-672e4414c30b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:22.528535 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5950ef-cf4f-4700-953e-672e4414c30b admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:22.592277 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf28bdf2-582f-4d08-9f70-2c6398655bc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:22.593080 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf28bdf2-582f-4d08-9f70-2c6398655bc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:22.593370 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf28bdf2-582f-4d08-9f70-2c6398655bc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:22.593711 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf28bdf2-582f-4d08-9f70-2c6398655bc3 admin admin] PATH_INFO: `/v3/projects/1b7c6506581244188949a54eddd6f6aa` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:22.594852 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf28bdf2-582f-4d08-9f70-2c6398655bc3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b7c6506581244188949a54eddd6f6aa)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:22.597458 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf28bdf2-582f-4d08-9f70-2c6398655bc3 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:22.646917 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-bf28bdf2-582f-4d08-9f70-2c6398655bc3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b7c6506581244188949a54eddd6f6aa', 'request_id': 'req-bf28bdf2-582f-4d08-9f70-2c6398655bc3'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:22.648069 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:22.654674 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-bf28bdf2-582f-4d08-9f70-2c6398655bc3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b7c6506581244188949a54eddd6f6aa', 'request_id': 'req-bf28bdf2-582f-4d08-9f70-2c6398655bc3'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:22.661110 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-bf28bdf2-582f-4d08-9f70-2c6398655bc3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b7c6506581244188949a54eddd6f6aa', 'request_id': 'req-bf28bdf2-582f-4d08-9f70-2c6398655bc3'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:22.673817 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-bf28bdf2-582f-4d08-9f70-2c6398655bc3 admin admin] The token cache is being invalidate because project 1b7c6506581244188949a54eddd6f6aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 15:38:22.674110 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-bf28bdf2-582f-4d08-9f70-2c6398655bc3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:22.674740 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-bf28bdf2-582f-4d08-9f70-2c6398655bc3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:22.676432 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1053/2119] 10.222.0.99 () {58 vars in 1232 bytes} [Thu Jul 14 15:38:22 2022] DELETE /identity/v3/projects/1b7c6506581244188949a54eddd6f6aa => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:38:22.681408 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a6ccab-476f-41ed-a158-523aaf30b991 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:22.681885 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a6ccab-476f-41ed-a158-523aaf30b991 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:22.682005 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a6ccab-476f-41ed-a158-523aaf30b991 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:22.700675 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-62a6ccab-476f-41ed-a158-523aaf30b991 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:38:22.701891 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-62a6ccab-476f-41ed-a158-523aaf30b991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:22.762663 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1067/2120] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:38:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:38:22.770884 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017be7da-9054-450e-8a19-436709c85413 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:22.771150 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017be7da-9054-450e-8a19-436709c85413 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:22.771735 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017be7da-9054-450e-8a19-436709c85413 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:22.789901 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-017be7da-9054-450e-8a19-436709c85413 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:38:22.791120 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-017be7da-9054-450e-8a19-436709c85413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:22.799871 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1054/2121] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:38:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:38:22.804228 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a6ccab-476f-41ed-a158-523aaf30b991 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:22.815181 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a6ccab-476f-41ed-a158-523aaf30b991 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:22.819837 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced7887d-ab43-4e1c-8f64-8fe618874702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:22.820497 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced7887d-ab43-4e1c-8f64-8fe618874702 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:22.820784 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced7887d-ab43-4e1c-8f64-8fe618874702 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:22.821062 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced7887d-ab43-4e1c-8f64-8fe618874702 admin admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:22.821798 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced7887d-ab43-4e1c-8f64-8fe618874702 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:38:22.822107 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced7887d-ab43-4e1c-8f64-8fe618874702 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:22.823735 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced7887d-ab43-4e1c-8f64-8fe618874702 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:22.835854 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1068/2122] 10.222.0.99 () {58 vars in 1152 bytes} [Thu Jul 14 15:38:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:38:22.840026 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017be7da-9054-450e-8a19-436709c85413 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:22.849597 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017be7da-9054-450e-8a19-436709c85413 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:22.854184 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1222f296-7075-437e-a7ce-c9c6d757bd4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:22.854847 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1222f296-7075-437e-a7ce-c9c6d757bd4f admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:22.855129 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1222f296-7075-437e-a7ce-c9c6d757bd4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:22.855409 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1222f296-7075-437e-a7ce-c9c6d757bd4f admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:22.856140 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1222f296-7075-437e-a7ce-c9c6d757bd4f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:22.858047 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1222f296-7075-437e-a7ce-c9c6d757bd4f admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:22.889699 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1055/2123] 10.222.0.99 () {60 vars in 1152 bytes} [Thu Jul 14 15:38:22 2022] POST /identity/v3/projects => generated 363 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:38:22.890998 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced7887d-ab43-4e1c-8f64-8fe618874702 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:22.904295 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced7887d-ab43-4e1c-8f64-8fe618874702 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:22.966068 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:22.978160 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2881017-a582-44dc-85b8-9c32b30341d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:22.978948 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2881017-a582-44dc-85b8-9c32b30341d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:22.979478 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2881017-a582-44dc-85b8-9c32b30341d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:22.980034 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2881017-a582-44dc-85b8-9c32b30341d5 admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:22.980937 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2881017-a582-44dc-85b8-9c32b30341d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:22.983134 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2881017-a582-44dc-85b8-9c32b30341d5 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:23.005296 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1069/2124] 10.222.0.99 () {60 vars in 1143 bytes} [Thu Jul 14 15:38:22 2022] POST /identity/v3/users => generated 282 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:38:23.009422 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1222f296-7075-437e-a7ce-c9c6d757bd4f admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:23.020797 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1222f296-7075-437e-a7ce-c9c6d757bd4f admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:23.025771 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034c12e8-a665-4d0a-8cad-dab175bc184c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:23.026445 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034c12e8-a665-4d0a-8cad-dab175bc184c admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:23.026734 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034c12e8-a665-4d0a-8cad-dab175bc184c admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:23.027015 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034c12e8-a665-4d0a-8cad-dab175bc184c admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:23.027785 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034c12e8-a665-4d0a-8cad-dab175bc184c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:23.029426 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034c12e8-a665-4d0a-8cad-dab175bc184c admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:23.042697 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1056/2125] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:38:23 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:38:23.047524 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2881017-a582-44dc-85b8-9c32b30341d5 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:23.059560 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2881017-a582-44dc-85b8-9c32b30341d5 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:23.065353 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65e6b1a-7fb8-4a77-b8b5-8a7551c2356d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:23.066055 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65e6b1a-7fb8-4a77-b8b5-8a7551c2356d admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:23.066346 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65e6b1a-7fb8-4a77-b8b5-8a7551c2356d admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:23.066630 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65e6b1a-7fb8-4a77-b8b5-8a7551c2356d admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:23.067386 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65e6b1a-7fb8-4a77-b8b5-8a7551c2356d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:23.069076 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65e6b1a-7fb8-4a77-b8b5-8a7551c2356d admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:23.080856 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1070/2126] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:38:23 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 15:38:23.085712 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034c12e8-a665-4d0a-8cad-dab175bc184c admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:23.096964 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034c12e8-a665-4d0a-8cad-dab175bc184c admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:23.101910 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f5f0bd-2849-4535-9b03-7d6552454399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:23.102858 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f5f0bd-2849-4535-9b03-7d6552454399 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:23.103232 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f5f0bd-2849-4535-9b03-7d6552454399 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:23.103580 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f5f0bd-2849-4535-9b03-7d6552454399 admin admin] PATH_INFO: `/v3/projects/0c6dccb5254b445fabe8750402403eaa/users/d9bfee9e48754dfa9c61dccef4856241/roles/b8c98fddc0fd4436b577fd1d3361bc38` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:23.121331 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f5f0bd-2849-4535-9b03-7d6552454399 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c6dccb5254b445fabe8750402403eaa, user_id=d9bfee9e48754dfa9c61dccef4856241, role_id=b8c98fddc0fd4436b577fd1d3361bc38)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:23.124194 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f5f0bd-2849-4535-9b03-7d6552454399 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:23.140874 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1057/2127] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 15:38:23 2022] PUT /identity/v3/projects/0c6dccb5254b445fabe8750402403eaa/users/d9bfee9e48754dfa9c61dccef4856241/roles/b8c98fddc0fd4436b577fd1d3361bc38 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:38:23.151708 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165aaac2-5f8b-4df3-b618-e5c62a54e161 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:23.151994 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165aaac2-5f8b-4df3-b618-e5c62a54e161 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:23.152281 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165aaac2-5f8b-4df3-b618-e5c62a54e161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:23.182205 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-165aaac2-5f8b-4df3-b618-e5c62a54e161 None None] MFA Rules not processed for user `d9bfee9e48754dfa9c61dccef4856241`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:38:23.183502 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-165aaac2-5f8b-4df3-b618-e5c62a54e161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:23.286841 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1071/2128] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:38:23 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:38:23.291369 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f5f0bd-2849-4535-9b03-7d6552454399 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:23.360208 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f5f0bd-2849-4535-9b03-7d6552454399 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:23.364852 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21988394-cc3d-4b99-98c4-ae73446d58ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:23.365509 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21988394-cc3d-4b99-98c4-ae73446d58ae admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:23.365796 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21988394-cc3d-4b99-98c4-ae73446d58ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:23.366074 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21988394-cc3d-4b99-98c4-ae73446d58ae admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:23.366773 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21988394-cc3d-4b99-98c4-ae73446d58ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:23.368746 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21988394-cc3d-4b99-98c4-ae73446d58ae admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:23.394688 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1058/2129] 10.222.0.99 () {60 vars in 1152 bytes} [Thu Jul 14 15:38:23 2022] POST /identity/v3/projects => generated 363 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:38:23.398807 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165aaac2-5f8b-4df3-b618-e5c62a54e161 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:23.410070 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165aaac2-5f8b-4df3-b618-e5c62a54e161 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:23.468380 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cb34e4-df35-4330-8833-b59722c22a15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:23.469062 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cb34e4-df35-4330-8833-b59722c22a15 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:23.469350 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cb34e4-df35-4330-8833-b59722c22a15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:23.469633 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cb34e4-df35-4330-8833-b59722c22a15 admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:23.471331 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cb34e4-df35-4330-8833-b59722c22a15 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:23.473374 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cb34e4-df35-4330-8833-b59722c22a15 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:23.493848 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1072/2130] 10.222.0.99 () {60 vars in 1143 bytes} [Thu Jul 14 15:38:23 2022] POST /identity/v3/users => generated 288 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:38:23.497800 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21988394-cc3d-4b99-98c4-ae73446d58ae admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:23.508320 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21988394-cc3d-4b99-98c4-ae73446d58ae admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:23.512837 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae47dac3-3b50-462e-a62f-796a5cd7889f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:23.513488 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae47dac3-3b50-462e-a62f-796a5cd7889f admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:23.513774 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae47dac3-3b50-462e-a62f-796a5cd7889f admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:23.514053 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae47dac3-3b50-462e-a62f-796a5cd7889f admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:23.514787 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae47dac3-3b50-462e-a62f-796a5cd7889f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:23.516565 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae47dac3-3b50-462e-a62f-796a5cd7889f admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:23.527882 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1059/2131] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:38:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:38:23.532034 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cb34e4-df35-4330-8833-b59722c22a15 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:23.544250 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cb34e4-df35-4330-8833-b59722c22a15 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:23.549343 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd481864-6056-48a2-bf5c-e0a5ab8a6ea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:23.550225 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd481864-6056-48a2-bf5c-e0a5ab8a6ea5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:23.550508 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd481864-6056-48a2-bf5c-e0a5ab8a6ea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:23.550785 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd481864-6056-48a2-bf5c-e0a5ab8a6ea5 admin admin] PATH_INFO: `/v3/projects/687203a2537f4495987309a64c0c08b3/users/1b8b0e060df144499006ce25cfab53da/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:23.566455 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd481864-6056-48a2-bf5c-e0a5ab8a6ea5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=687203a2537f4495987309a64c0c08b3, user_id=1b8b0e060df144499006ce25cfab53da, role_id=d0f6ba1314f14d9f9c816bdb5d0c3aac)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:23.569189 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd481864-6056-48a2-bf5c-e0a5ab8a6ea5 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:23.586085 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1073/2132] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 15:38:23 2022] PUT /identity/v3/projects/687203a2537f4495987309a64c0c08b3/users/1b8b0e060df144499006ce25cfab53da/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:38:23.590370 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae47dac3-3b50-462e-a62f-796a5cd7889f admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:23.604086 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae47dac3-3b50-462e-a62f-796a5cd7889f admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:23.666887 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:23.682464 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58961c38-f7db-4c78-ac4e-d9ff38ab9ff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:23.683223 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58961c38-f7db-4c78-ac4e-d9ff38ab9ff7 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:23.683942 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58961c38-f7db-4c78-ac4e-d9ff38ab9ff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:23.684309 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58961c38-f7db-4c78-ac4e-d9ff38ab9ff7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:23.685657 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58961c38-f7db-4c78-ac4e-d9ff38ab9ff7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:23.687440 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58961c38-f7db-4c78-ac4e-d9ff38ab9ff7 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:23.699175 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1060/2133] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:38:23 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:38:23.703477 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd481864-6056-48a2-bf5c-e0a5ab8a6ea5 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:23.715364 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd481864-6056-48a2-bf5c-e0a5ab8a6ea5 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:23.720289 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9679f1-ca7d-4d59-b028-e3af7a68e8d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:23.721209 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9679f1-ca7d-4d59-b028-e3af7a68e8d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:23.721500 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9679f1-ca7d-4d59-b028-e3af7a68e8d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:23.721787 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9679f1-ca7d-4d59-b028-e3af7a68e8d0 admin admin] PATH_INFO: `/v3/projects/687203a2537f4495987309a64c0c08b3/users/1b8b0e060df144499006ce25cfab53da/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:23.723861 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9679f1-ca7d-4d59-b028-e3af7a68e8d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=687203a2537f4495987309a64c0c08b3, user_id=1b8b0e060df144499006ce25cfab53da, role_id=d0f6ba1314f14d9f9c816bdb5d0c3aac)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:23.726549 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9679f1-ca7d-4d59-b028-e3af7a68e8d0 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:23.737145 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1074/2134] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 15:38:23 2022] PUT /identity/v3/projects/687203a2537f4495987309a64c0c08b3/users/1b8b0e060df144499006ce25cfab53da/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:38:23.748348 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb7715f-514d-4930-aa21-87e7f654e30e None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:23.748651 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb7715f-514d-4930-aa21-87e7f654e30e None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:23.748940 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb7715f-514d-4930-aa21-87e7f654e30e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:23.780405 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-0fb7715f-514d-4930-aa21-87e7f654e30e None None] MFA Rules not processed for user `1b8b0e060df144499006ce25cfab53da`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:38:23.781644 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0fb7715f-514d-4930-aa21-87e7f654e30e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:23.849147 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1061/2135] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:38:23 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:38:23.860365 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9679f1-ca7d-4d59-b028-e3af7a68e8d0 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:23.862703 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-df9679f1-ca7d-4d59-b028-e3af7a68e8d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:23.878102 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:23.922523 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9679f1-ca7d-4d59-b028-e3af7a68e8d0 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:23.926194 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a639d3f-ebc4-4818-8bdc-13775f57a8af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3e1a2b83f3d4d88ba5084dfaffd22f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:23.926840 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a639d3f-ebc4-4818-8bdc-13775f57a8af service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:23.927127 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a639d3f-ebc4-4818-8bdc-13775f57a8af service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:23.927408 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a639d3f-ebc4-4818-8bdc-13775f57a8af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:23.936473 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a639d3f-ebc4-4818-8bdc-13775f57a8af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:23.937942 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a639d3f-ebc4-4818-8bdc-13775f57a8af service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:23.948103 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1075/2136] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:38:23 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:38:24.022707 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb7715f-514d-4930-aa21-87e7f654e30e None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:24.032228 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb7715f-514d-4930-aa21-87e7f654e30e None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:24.036235 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375ecf58-92a5-4272-bc5c-878cd74ce699 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3e1a2b83f3d4d88ba5084dfaffd22f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:24.036881 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375ecf58-92a5-4272-bc5c-878cd74ce699 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:24.037173 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375ecf58-92a5-4272-bc5c-878cd74ce699 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:24.037463 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375ecf58-92a5-4272-bc5c-878cd74ce699 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:24.097257 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375ecf58-92a5-4272-bc5c-878cd74ce699 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:24.098710 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375ecf58-92a5-4272-bc5c-878cd74ce699 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:24.107681 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1062/2137] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:38:24 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:38:27.269921 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a639d3f-ebc4-4818-8bdc-13775f57a8af service cinder] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:27.402425 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a639d3f-ebc4-4818-8bdc-13775f57a8af service cinder] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:27.416057 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa30109a-3db8-43b6-9f5c-29dbc450c67e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:27.417473 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa30109a-3db8-43b6-9f5c-29dbc450c67e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:27.417767 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa30109a-3db8-43b6-9f5c-29dbc450c67e admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:27.418043 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa30109a-3db8-43b6-9f5c-29dbc450c67e admin admin] PATH_INFO: `/v3/users/d9bfee9e48754dfa9c61dccef4856241` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:27.419518 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa30109a-3db8-43b6-9f5c-29dbc450c67e admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9bfee9e48754dfa9c61dccef4856241)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:27.421735 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa30109a-3db8-43b6-9f5c-29dbc450c67e admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:27.521875 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-aa30109a-3db8-43b6-9f5c-29dbc450c67e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9bfee9e48754dfa9c61dccef4856241'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:27.535070 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-aa30109a-3db8-43b6-9f5c-29dbc450c67e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9bfee9e48754dfa9c61dccef4856241'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:27.556697 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-aa30109a-3db8-43b6-9f5c-29dbc450c67e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9bfee9e48754dfa9c61dccef4856241'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:27.557480 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-aa30109a-3db8-43b6-9f5c-29dbc450c67e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9bfee9e48754dfa9c61dccef4856241'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:27.558516 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-aa30109a-3db8-43b6-9f5c-29dbc450c67e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9bfee9e48754dfa9c61dccef4856241'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:27.575471 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1076/2138] 10.222.0.99 () {58 vars in 1223 bytes} [Thu Jul 14 15:38:27 2022] DELETE /identity/v3/users/d9bfee9e48754dfa9c61dccef4856241 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:38:27.580877 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b73db89-9303-402a-8448-b344781b61f2 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:27.581414 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b73db89-9303-402a-8448-b344781b61f2 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:27.581414 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b73db89-9303-402a-8448-b344781b61f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:27.606534 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-8b73db89-9303-402a-8448-b344781b61f2 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:38:27.608838 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-8b73db89-9303-402a-8448-b344781b61f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:27.689408 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1063/2139] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:38:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:38:27.709338 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa30109a-3db8-43b6-9f5c-29dbc450c67e admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:27.712876 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-aa30109a-3db8-43b6-9f5c-29dbc450c67e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:27.765340 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:27.787188 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa30109a-3db8-43b6-9f5c-29dbc450c67e admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:27.791393 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca0f88a-08f3-44d7-8d4f-cff209a88179 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:27.792079 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca0f88a-08f3-44d7-8d4f-cff209a88179 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:27.792365 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca0f88a-08f3-44d7-8d4f-cff209a88179 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:27.792646 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca0f88a-08f3-44d7-8d4f-cff209a88179 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:27.802892 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca0f88a-08f3-44d7-8d4f-cff209a88179 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:27.804554 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca0f88a-08f3-44d7-8d4f-cff209a88179 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:27.815051 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1077/2140] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:38:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:38:28.135234 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b73db89-9303-402a-8448-b344781b61f2 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:28.137949 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-8b73db89-9303-402a-8448-b344781b61f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:28.173399 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b73db89-9303-402a-8448-b344781b61f2 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:28.188068 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ee6cb8-be54-44f8-8df9-9a01fc25640a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:28.189259 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ee6cb8-be54-44f8-8df9-9a01fc25640a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:28.190322 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ee6cb8-be54-44f8-8df9-9a01fc25640a admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:28.190617 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ee6cb8-be54-44f8-8df9-9a01fc25640a admin admin] PATH_INFO: `/v3/projects/0c6dccb5254b445fabe8750402403eaa` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:28.192406 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ee6cb8-be54-44f8-8df9-9a01fc25640a admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c6dccb5254b445fabe8750402403eaa)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:28.194579 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ee6cb8-be54-44f8-8df9-9a01fc25640a admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:28.197696 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:28.264149 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-13ee6cb8-be54-44f8-8df9-9a01fc25640a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c6dccb5254b445fabe8750402403eaa', 'request_id': 'req-13ee6cb8-be54-44f8-8df9-9a01fc25640a'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:28.274636 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-13ee6cb8-be54-44f8-8df9-9a01fc25640a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c6dccb5254b445fabe8750402403eaa', 'request_id': 'req-13ee6cb8-be54-44f8-8df9-9a01fc25640a'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:28.282126 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-13ee6cb8-be54-44f8-8df9-9a01fc25640a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c6dccb5254b445fabe8750402403eaa', 'request_id': 'req-13ee6cb8-be54-44f8-8df9-9a01fc25640a'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:28.302788 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-13ee6cb8-be54-44f8-8df9-9a01fc25640a admin admin] The token cache is being invalidate because project 0c6dccb5254b445fabe8750402403eaa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 15:38:28.303158 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-13ee6cb8-be54-44f8-8df9-9a01fc25640a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:28.303880 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-13ee6cb8-be54-44f8-8df9-9a01fc25640a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:28.305747 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1064/2141] 10.222.0.99 () {58 vars in 1232 bytes} [Thu Jul 14 15:38:28 2022] DELETE /identity/v3/projects/0c6dccb5254b445fabe8750402403eaa => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:38:28.309844 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca0f88a-08f3-44d7-8d4f-cff209a88179 service neutron] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:28.327637 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca0f88a-08f3-44d7-8d4f-cff209a88179 service neutron] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:28.395828 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892a593e-782c-4e0d-b07e-4437c3e48601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:28.397336 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892a593e-782c-4e0d-b07e-4437c3e48601 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:28.397823 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892a593e-782c-4e0d-b07e-4437c3e48601 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:28.398403 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892a593e-782c-4e0d-b07e-4437c3e48601 admin admin] PATH_INFO: `/v3/users/1b8b0e060df144499006ce25cfab53da` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:28.400460 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892a593e-782c-4e0d-b07e-4437c3e48601 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b8b0e060df144499006ce25cfab53da)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:28.403710 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892a593e-782c-4e0d-b07e-4437c3e48601 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:28.464918 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-892a593e-782c-4e0d-b07e-4437c3e48601 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b8b0e060df144499006ce25cfab53da'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:28.473478 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-892a593e-782c-4e0d-b07e-4437c3e48601 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b8b0e060df144499006ce25cfab53da'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:28.487258 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-892a593e-782c-4e0d-b07e-4437c3e48601 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b8b0e060df144499006ce25cfab53da'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:28.487994 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-892a593e-782c-4e0d-b07e-4437c3e48601 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b8b0e060df144499006ce25cfab53da'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:28.488628 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-892a593e-782c-4e0d-b07e-4437c3e48601 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b8b0e060df144499006ce25cfab53da'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:28.503303 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1078/2142] 10.222.0.99 () {58 vars in 1223 bytes} [Thu Jul 14 15:38:28 2022] DELETE /identity/v3/users/1b8b0e060df144499006ce25cfab53da => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 15:38:28.800015 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ee6cb8-be54-44f8-8df9-9a01fc25640a admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:28.801551 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-13ee6cb8-be54-44f8-8df9-9a01fc25640a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:28.815467 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ee6cb8-be54-44f8-8df9-9a01fc25640a admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:28.882829 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3a6490-0eac-407e-a35d-eb5b80c55ed9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:28.884125 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3a6490-0eac-407e-a35d-eb5b80c55ed9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:28.884418 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3a6490-0eac-407e-a35d-eb5b80c55ed9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:28.884717 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3a6490-0eac-407e-a35d-eb5b80c55ed9 admin admin] PATH_INFO: `/v3/projects/687203a2537f4495987309a64c0c08b3` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:28.885990 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3a6490-0eac-407e-a35d-eb5b80c55ed9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=687203a2537f4495987309a64c0c08b3)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:28.887983 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3a6490-0eac-407e-a35d-eb5b80c55ed9 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:28.937253 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-1a3a6490-0eac-407e-a35d-eb5b80c55ed9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '687203a2537f4495987309a64c0c08b3', 'request_id': 'req-1a3a6490-0eac-407e-a35d-eb5b80c55ed9'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:28.944093 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-1a3a6490-0eac-407e-a35d-eb5b80c55ed9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '687203a2537f4495987309a64c0c08b3', 'request_id': 'req-1a3a6490-0eac-407e-a35d-eb5b80c55ed9'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:28.949628 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-1a3a6490-0eac-407e-a35d-eb5b80c55ed9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '687203a2537f4495987309a64c0c08b3', 'request_id': 'req-1a3a6490-0eac-407e-a35d-eb5b80c55ed9'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:28.962919 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-1a3a6490-0eac-407e-a35d-eb5b80c55ed9 admin admin] The token cache is being invalidate because project 687203a2537f4495987309a64c0c08b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 15:38:28.963208 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-1a3a6490-0eac-407e-a35d-eb5b80c55ed9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:28.963900 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-1a3a6490-0eac-407e-a35d-eb5b80c55ed9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:28.965458 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1065/2143] 10.222.0.99 () {58 vars in 1232 bytes} [Thu Jul 14 15:38:28 2022] DELETE /identity/v3/projects/687203a2537f4495987309a64c0c08b3 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:38:28.970841 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e608c0-5fc4-487e-991a-a3436aac52d6 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:28.971274 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e608c0-5fc4-487e-991a-a3436aac52d6 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:28.971693 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e608c0-5fc4-487e-991a-a3436aac52d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:28.990236 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-c3e608c0-5fc4-487e-991a-a3436aac52d6 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:38:28.991606 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-c3e608c0-5fc4-487e-991a-a3436aac52d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:29.052479 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:29.063747 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1079/2144] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:38:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:38:29.072696 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce800d9-417d-4c49-aea2-f294a8197a58 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:29.072992 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce800d9-417d-4c49-aea2-f294a8197a58 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:29.073278 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce800d9-417d-4c49-aea2-f294a8197a58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:29.093250 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-1ce800d9-417d-4c49-aea2-f294a8197a58 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:38:29.094468 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-1ce800d9-417d-4c49-aea2-f294a8197a58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:29.102389 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1066/2145] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:38:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:38:29.106732 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e608c0-5fc4-487e-991a-a3436aac52d6 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:29.118590 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e608c0-5fc4-487e-991a-a3436aac52d6 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:29.123330 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c90d48-7514-489c-9ebb-efa976de46a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:29.124037 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c90d48-7514-489c-9ebb-efa976de46a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:29.124328 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c90d48-7514-489c-9ebb-efa976de46a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:29.124612 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c90d48-7514-489c-9ebb-efa976de46a4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:29.125368 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c90d48-7514-489c-9ebb-efa976de46a4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:38:29.125753 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c90d48-7514-489c-9ebb-efa976de46a4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:29.128464 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c90d48-7514-489c-9ebb-efa976de46a4 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:29.140693 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1080/2146] 10.222.0.99 () {58 vars in 1152 bytes} [Thu Jul 14 15:38:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:38:29.145991 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce800d9-417d-4c49-aea2-f294a8197a58 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:29.155724 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce800d9-417d-4c49-aea2-f294a8197a58 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:29.160320 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37de3375-e774-4517-9990-c16b51f75e7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:29.160970 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37de3375-e774-4517-9990-c16b51f75e7a admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:29.161255 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37de3375-e774-4517-9990-c16b51f75e7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:29.161537 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37de3375-e774-4517-9990-c16b51f75e7a admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:29.162260 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37de3375-e774-4517-9990-c16b51f75e7a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:29.164211 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37de3375-e774-4517-9990-c16b51f75e7a admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:29.177636 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:29.195180 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1067/2147] 10.222.0.99 () {60 vars in 1152 bytes} [Thu Jul 14 15:38:29 2022] POST /identity/v3/projects => generated 387 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:38:29.197030 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c90d48-7514-489c-9ebb-efa976de46a4 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:29.209786 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c90d48-7514-489c-9ebb-efa976de46a4 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:29.270772 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa78e29a-983e-4928-a5d6-dab5accc65d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:29.271742 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa78e29a-983e-4928-a5d6-dab5accc65d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:29.272038 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa78e29a-983e-4928-a5d6-dab5accc65d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:29.272386 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa78e29a-983e-4928-a5d6-dab5accc65d5 admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:29.273028 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa78e29a-983e-4928-a5d6-dab5accc65d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:29.275029 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa78e29a-983e-4928-a5d6-dab5accc65d5 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:29.297313 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1081/2148] 10.222.0.99 () {60 vars in 1143 bytes} [Thu Jul 14 15:38:29 2022] POST /identity/v3/users => generated 294 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:38:29.300280 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37de3375-e774-4517-9990-c16b51f75e7a admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:29.310619 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37de3375-e774-4517-9990-c16b51f75e7a admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:29.316575 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2dd6db-9d11-42c0-abd0-921a3d5bc1da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:29.317125 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2dd6db-9d11-42c0-abd0-921a3d5bc1da admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:29.317548 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2dd6db-9d11-42c0-abd0-921a3d5bc1da admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:29.317690 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2dd6db-9d11-42c0-abd0-921a3d5bc1da admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:29.318644 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2dd6db-9d11-42c0-abd0-921a3d5bc1da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:29.320051 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2dd6db-9d11-42c0-abd0-921a3d5bc1da admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:29.332075 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1068/2149] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:38:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:38:29.336391 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa78e29a-983e-4928-a5d6-dab5accc65d5 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:29.346359 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa78e29a-983e-4928-a5d6-dab5accc65d5 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:29.351039 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83068b73-811c-4d78-b996-5b46f0784af8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:29.351748 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83068b73-811c-4d78-b996-5b46f0784af8 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:29.352045 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83068b73-811c-4d78-b996-5b46f0784af8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:29.352330 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83068b73-811c-4d78-b996-5b46f0784af8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:29.353069 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83068b73-811c-4d78-b996-5b46f0784af8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:29.354682 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83068b73-811c-4d78-b996-5b46f0784af8 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:29.365576 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1082/2150] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:38:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:38:29.369384 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2dd6db-9d11-42c0-abd0-921a3d5bc1da admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:29.380894 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2dd6db-9d11-42c0-abd0-921a3d5bc1da admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:29.385873 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc27dc4-6563-47d8-b6e3-3df7b678ddee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:29.386796 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc27dc4-6563-47d8-b6e3-3df7b678ddee admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:29.387084 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc27dc4-6563-47d8-b6e3-3df7b678ddee admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:29.387365 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc27dc4-6563-47d8-b6e3-3df7b678ddee admin admin] PATH_INFO: `/v3/projects/4babcbcfdc21440badf2bb5897cd6096/users/ef96411e7a1f4ec096c08e0abfe31664/roles/b8c98fddc0fd4436b577fd1d3361bc38` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:29.403438 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc27dc4-6563-47d8-b6e3-3df7b678ddee admin admin] RBAC: Authorizing `identity:create_grant(project_id=4babcbcfdc21440badf2bb5897cd6096, user_id=ef96411e7a1f4ec096c08e0abfe31664, role_id=b8c98fddc0fd4436b577fd1d3361bc38)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:29.406549 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc27dc4-6563-47d8-b6e3-3df7b678ddee admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:29.420884 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1069/2151] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 15:38:29 2022] PUT /identity/v3/projects/4babcbcfdc21440badf2bb5897cd6096/users/ef96411e7a1f4ec096c08e0abfe31664/roles/b8c98fddc0fd4436b577fd1d3361bc38 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:38:29.432005 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18eaeae-b445-4b16-97ac-07992f58584a None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:29.432302 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18eaeae-b445-4b16-97ac-07992f58584a None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:29.432585 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18eaeae-b445-4b16-97ac-07992f58584a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:29.464411 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-f18eaeae-b445-4b16-97ac-07992f58584a None None] MFA Rules not processed for user `ef96411e7a1f4ec096c08e0abfe31664`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:38:29.465565 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-f18eaeae-b445-4b16-97ac-07992f58584a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:29.552756 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1083/2152] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:38:29 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:38:29.558584 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc27dc4-6563-47d8-b6e3-3df7b678ddee admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:29.628368 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc27dc4-6563-47d8-b6e3-3df7b678ddee admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:29.632999 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5189a437-a774-4373-b221-ba43f70efe8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:29.633664 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5189a437-a774-4373-b221-ba43f70efe8f admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:29.633955 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5189a437-a774-4373-b221-ba43f70efe8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:29.634237 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5189a437-a774-4373-b221-ba43f70efe8f admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:29.634934 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5189a437-a774-4373-b221-ba43f70efe8f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:29.636905 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5189a437-a774-4373-b221-ba43f70efe8f admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:29.663290 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1070/2153] 10.222.0.99 () {60 vars in 1152 bytes} [Thu Jul 14 15:38:29 2022] POST /identity/v3/projects => generated 387 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 15:38:29.667334 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18eaeae-b445-4b16-97ac-07992f58584a None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:29.679227 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18eaeae-b445-4b16-97ac-07992f58584a None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:29.729873 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:29.740653 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58149172-6fa2-43f7-87a9-7fd10eed3119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:29.741333 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58149172-6fa2-43f7-87a9-7fd10eed3119 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:29.741623 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58149172-6fa2-43f7-87a9-7fd10eed3119 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:29.741907 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58149172-6fa2-43f7-87a9-7fd10eed3119 admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:29.742608 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58149172-6fa2-43f7-87a9-7fd10eed3119 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:29.744569 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58149172-6fa2-43f7-87a9-7fd10eed3119 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:29.768051 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1084/2154] 10.222.0.99 () {60 vars in 1143 bytes} [Thu Jul 14 15:38:29 2022] POST /identity/v3/users => generated 300 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:38:29.772468 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5189a437-a774-4373-b221-ba43f70efe8f admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:29.784138 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5189a437-a774-4373-b221-ba43f70efe8f admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:29.789296 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde43f87-26ee-4da6-a7cf-ae51d00e18dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:29.790000 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde43f87-26ee-4da6-a7cf-ae51d00e18dc admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:29.790288 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde43f87-26ee-4da6-a7cf-ae51d00e18dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:29.790580 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde43f87-26ee-4da6-a7cf-ae51d00e18dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:29.791341 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde43f87-26ee-4da6-a7cf-ae51d00e18dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:29.793009 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde43f87-26ee-4da6-a7cf-ae51d00e18dc admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:29.805993 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1071/2155] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:38:29 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:38:29.810195 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58149172-6fa2-43f7-87a9-7fd10eed3119 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:29.820229 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58149172-6fa2-43f7-87a9-7fd10eed3119 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:29.824969 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191a9588-a316-44c5-8086-d5b0f4f8f3c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:29.825848 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191a9588-a316-44c5-8086-d5b0f4f8f3c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:29.826135 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191a9588-a316-44c5-8086-d5b0f4f8f3c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:29.826415 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191a9588-a316-44c5-8086-d5b0f4f8f3c8 admin admin] PATH_INFO: `/v3/projects/fb323527afa544d7b5030850907a23ef/users/643693c58dd54142ba24c175443f2c67/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:29.841674 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191a9588-a316-44c5-8086-d5b0f4f8f3c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb323527afa544d7b5030850907a23ef, user_id=643693c58dd54142ba24c175443f2c67, role_id=d0f6ba1314f14d9f9c816bdb5d0c3aac)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:29.844442 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191a9588-a316-44c5-8086-d5b0f4f8f3c8 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:29.858096 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1085/2156] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 15:38:29 2022] PUT /identity/v3/projects/fb323527afa544d7b5030850907a23ef/users/643693c58dd54142ba24c175443f2c67/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:38:29.862622 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde43f87-26ee-4da6-a7cf-ae51d00e18dc admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:29.872513 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde43f87-26ee-4da6-a7cf-ae51d00e18dc admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:29.929528 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce3cb66-55a4-4fe0-b16b-0d41c07851e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:29.930198 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce3cb66-55a4-4fe0-b16b-0d41c07851e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:29.930496 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce3cb66-55a4-4fe0-b16b-0d41c07851e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:29.930777 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce3cb66-55a4-4fe0-b16b-0d41c07851e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:29.931511 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce3cb66-55a4-4fe0-b16b-0d41c07851e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:29.933142 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce3cb66-55a4-4fe0-b16b-0d41c07851e4 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:29.943199 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1072/2157] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:38:29 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 15:38:29.947398 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191a9588-a316-44c5-8086-d5b0f4f8f3c8 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:29.958165 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191a9588-a316-44c5-8086-d5b0f4f8f3c8 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:29.962859 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f43346d-de4d-402f-b97c-e596b461488f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:29.963772 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f43346d-de4d-402f-b97c-e596b461488f admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:29.964059 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f43346d-de4d-402f-b97c-e596b461488f admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:29.964332 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f43346d-de4d-402f-b97c-e596b461488f admin admin] PATH_INFO: `/v3/projects/fb323527afa544d7b5030850907a23ef/users/643693c58dd54142ba24c175443f2c67/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:29.966300 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f43346d-de4d-402f-b97c-e596b461488f admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb323527afa544d7b5030850907a23ef, user_id=643693c58dd54142ba24c175443f2c67, role_id=d0f6ba1314f14d9f9c816bdb5d0c3aac)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:29.968967 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f43346d-de4d-402f-b97c-e596b461488f admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:29.980333 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1086/2158] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 15:38:29 2022] PUT /identity/v3/projects/fb323527afa544d7b5030850907a23ef/users/643693c58dd54142ba24c175443f2c67/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:38:29.992785 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735476c7-04dc-48b2-ae03-024371c23089 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:29.993173 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735476c7-04dc-48b2-ae03-024371c23089 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:29.993536 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735476c7-04dc-48b2-ae03-024371c23089 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:30.023123 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-735476c7-04dc-48b2-ae03-024371c23089 None None] MFA Rules not processed for user `643693c58dd54142ba24c175443f2c67`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:38:30.024389 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-735476c7-04dc-48b2-ae03-024371c23089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:30.042554 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:30.112325 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1073/2159] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:38:29 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:38:30.123345 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f43346d-de4d-402f-b97c-e596b461488f admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:30.125200 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-1f43346d-de4d-402f-b97c-e596b461488f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:30.185467 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f43346d-de4d-402f-b97c-e596b461488f admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:30.190861 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70763b9-fb29-43ca-9524-f11ceed42b5e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3e1a2b83f3d4d88ba5084dfaffd22f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:30.191988 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70763b9-fb29-43ca-9524-f11ceed42b5e service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:30.192513 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70763b9-fb29-43ca-9524-f11ceed42b5e service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:30.192819 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70763b9-fb29-43ca-9524-f11ceed42b5e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:30.205445 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70763b9-fb29-43ca-9524-f11ceed42b5e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:30.206962 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70763b9-fb29-43ca-9524-f11ceed42b5e service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:30.216638 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1087/2160] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:38:30 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:38:30.281330 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735476c7-04dc-48b2-ae03-024371c23089 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:30.344509 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735476c7-04dc-48b2-ae03-024371c23089 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:30.349430 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39759cd-f1d8-4fd2-a74e-c039b1c87515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:30.350299 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39759cd-f1d8-4fd2-a74e-c039b1c87515 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:30.350584 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39759cd-f1d8-4fd2-a74e-c039b1c87515 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:30.350862 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39759cd-f1d8-4fd2-a74e-c039b1c87515 admin admin] PATH_INFO: `/v3/users/ef96411e7a1f4ec096c08e0abfe31664` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:30.352120 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39759cd-f1d8-4fd2-a74e-c039b1c87515 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef96411e7a1f4ec096c08e0abfe31664)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:30.354045 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39759cd-f1d8-4fd2-a74e-c039b1c87515 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:30.402004 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-f39759cd-f1d8-4fd2-a74e-c039b1c87515 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef96411e7a1f4ec096c08e0abfe31664'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:30.410705 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-f39759cd-f1d8-4fd2-a74e-c039b1c87515 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef96411e7a1f4ec096c08e0abfe31664'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:30.422818 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-f39759cd-f1d8-4fd2-a74e-c039b1c87515 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef96411e7a1f4ec096c08e0abfe31664'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:30.424238 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-f39759cd-f1d8-4fd2-a74e-c039b1c87515 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef96411e7a1f4ec096c08e0abfe31664'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:30.424889 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-f39759cd-f1d8-4fd2-a74e-c039b1c87515 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef96411e7a1f4ec096c08e0abfe31664'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:30.437197 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1074/2161] 10.222.0.99 () {58 vars in 1223 bytes} [Thu Jul 14 15:38:30 2022] DELETE /identity/v3/users/ef96411e7a1f4ec096c08e0abfe31664 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:38:30.441674 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba56137-db7c-4901-aa3c-a71266508cdc None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:30.442151 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba56137-db7c-4901-aa3c-a71266508cdc None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:30.442252 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba56137-db7c-4901-aa3c-a71266508cdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:30.459834 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-2ba56137-db7c-4901-aa3c-a71266508cdc None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:38:30.461053 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-2ba56137-db7c-4901-aa3c-a71266508cdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:30.520550 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1088/2162] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:38:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:38:30.530484 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39759cd-f1d8-4fd2-a74e-c039b1c87515 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:30.531934 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f39759cd-f1d8-4fd2-a74e-c039b1c87515 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:30.568614 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:30.593266 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39759cd-f1d8-4fd2-a74e-c039b1c87515 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:30.598368 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15de2b5-5889-47b3-8cff-9a19da04398b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:30.599029 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15de2b5-5889-47b3-8cff-9a19da04398b service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:30.599313 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15de2b5-5889-47b3-8cff-9a19da04398b service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:30.599586 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15de2b5-5889-47b3-8cff-9a19da04398b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:30.609115 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15de2b5-5889-47b3-8cff-9a19da04398b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:30.610553 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15de2b5-5889-47b3-8cff-9a19da04398b service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:30.623769 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1075/2163] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:38:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:38:30.902303 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba56137-db7c-4901-aa3c-a71266508cdc None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:30.903790 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-2ba56137-db7c-4901-aa3c-a71266508cdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:30.916542 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba56137-db7c-4901-aa3c-a71266508cdc None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:30.921177 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc8a24a-9347-41aa-83f1-d3a5da2d95b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:30.922034 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc8a24a-9347-41aa-83f1-d3a5da2d95b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:30.922330 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc8a24a-9347-41aa-83f1-d3a5da2d95b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:30.922626 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc8a24a-9347-41aa-83f1-d3a5da2d95b2 admin admin] PATH_INFO: `/v3/projects/4babcbcfdc21440badf2bb5897cd6096` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:30.923897 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc8a24a-9347-41aa-83f1-d3a5da2d95b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4babcbcfdc21440badf2bb5897cd6096)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:30.925997 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc8a24a-9347-41aa-83f1-d3a5da2d95b2 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:30.970217 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:30.974863 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-9cc8a24a-9347-41aa-83f1-d3a5da2d95b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4babcbcfdc21440badf2bb5897cd6096', 'request_id': 'req-9cc8a24a-9347-41aa-83f1-d3a5da2d95b2'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:30.982194 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-9cc8a24a-9347-41aa-83f1-d3a5da2d95b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4babcbcfdc21440badf2bb5897cd6096', 'request_id': 'req-9cc8a24a-9347-41aa-83f1-d3a5da2d95b2'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:30.989057 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-9cc8a24a-9347-41aa-83f1-d3a5da2d95b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4babcbcfdc21440badf2bb5897cd6096', 'request_id': 'req-9cc8a24a-9347-41aa-83f1-d3a5da2d95b2'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:31.003116 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-9cc8a24a-9347-41aa-83f1-d3a5da2d95b2 admin admin] The token cache is being invalidate because project 4babcbcfdc21440badf2bb5897cd6096 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 15:38:31.003412 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-9cc8a24a-9347-41aa-83f1-d3a5da2d95b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:31.004081 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-9cc8a24a-9347-41aa-83f1-d3a5da2d95b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:31.005667 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1089/2164] 10.222.0.99 () {58 vars in 1232 bytes} [Thu Jul 14 15:38:30 2022] DELETE /identity/v3/projects/4babcbcfdc21440badf2bb5897cd6096 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:38:31.009692 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15de2b5-5889-47b3-8cff-9a19da04398b service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:31.020055 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15de2b5-5889-47b3-8cff-9a19da04398b service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:31.077650 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebc89bf-f6fc-4522-8fd9-7a188f248f02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:31.078520 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebc89bf-f6fc-4522-8fd9-7a188f248f02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:31.078807 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebc89bf-f6fc-4522-8fd9-7a188f248f02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:31.079086 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebc89bf-f6fc-4522-8fd9-7a188f248f02 admin admin] PATH_INFO: `/v3/users/643693c58dd54142ba24c175443f2c67` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:31.080813 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebc89bf-f6fc-4522-8fd9-7a188f248f02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=643693c58dd54142ba24c175443f2c67)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:31.082746 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebc89bf-f6fc-4522-8fd9-7a188f248f02 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:31.133500 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-3ebc89bf-f6fc-4522-8fd9-7a188f248f02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '643693c58dd54142ba24c175443f2c67'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:31.141536 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-3ebc89bf-f6fc-4522-8fd9-7a188f248f02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '643693c58dd54142ba24c175443f2c67'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:31.154436 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-3ebc89bf-f6fc-4522-8fd9-7a188f248f02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '643693c58dd54142ba24c175443f2c67'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:31.155172 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-3ebc89bf-f6fc-4522-8fd9-7a188f248f02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '643693c58dd54142ba24c175443f2c67'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:31.155817 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-3ebc89bf-f6fc-4522-8fd9-7a188f248f02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '643693c58dd54142ba24c175443f2c67'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:31.168983 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1076/2165] 10.222.0.99 () {58 vars in 1223 bytes} [Thu Jul 14 15:38:31 2022] DELETE /identity/v3/users/643693c58dd54142ba24c175443f2c67 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:38:31.445022 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc8a24a-9347-41aa-83f1-d3a5da2d95b2 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:31.446468 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-9cc8a24a-9347-41aa-83f1-d3a5da2d95b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:31.459699 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc8a24a-9347-41aa-83f1-d3a5da2d95b2 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:31.519377 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36426dc-8f78-4e8b-8f83-f432241e66a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:31.520190 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36426dc-8f78-4e8b-8f83-f432241e66a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:31.520508 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36426dc-8f78-4e8b-8f83-f432241e66a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:31.520791 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36426dc-8f78-4e8b-8f83-f432241e66a0 admin admin] PATH_INFO: `/v3/projects/fb323527afa544d7b5030850907a23ef` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:31.522248 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36426dc-8f78-4e8b-8f83-f432241e66a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb323527afa544d7b5030850907a23ef)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:31.524239 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36426dc-8f78-4e8b-8f83-f432241e66a0 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:31.575517 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-c36426dc-8f78-4e8b-8f83-f432241e66a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb323527afa544d7b5030850907a23ef', 'request_id': 'req-c36426dc-8f78-4e8b-8f83-f432241e66a0'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:31.582578 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-c36426dc-8f78-4e8b-8f83-f432241e66a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb323527afa544d7b5030850907a23ef', 'request_id': 'req-c36426dc-8f78-4e8b-8f83-f432241e66a0'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:31.588509 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-c36426dc-8f78-4e8b-8f83-f432241e66a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb323527afa544d7b5030850907a23ef', 'request_id': 'req-c36426dc-8f78-4e8b-8f83-f432241e66a0'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:31.602176 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-c36426dc-8f78-4e8b-8f83-f432241e66a0 admin admin] The token cache is being invalidate because project fb323527afa544d7b5030850907a23ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 15:38:31.602467 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-c36426dc-8f78-4e8b-8f83-f432241e66a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:31.603111 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-c36426dc-8f78-4e8b-8f83-f432241e66a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:31.604686 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1090/2166] 10.222.0.99 () {58 vars in 1232 bytes} [Thu Jul 14 15:38:31 2022] DELETE /identity/v3/projects/fb323527afa544d7b5030850907a23ef => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:38:31.609641 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81532e21-e978-4169-86fb-5f96c0667c8a None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:31.609938 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81532e21-e978-4169-86fb-5f96c0667c8a None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:31.610230 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81532e21-e978-4169-86fb-5f96c0667c8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:31.629565 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-81532e21-e978-4169-86fb-5f96c0667c8a None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:38:31.630792 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-81532e21-e978-4169-86fb-5f96c0667c8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:31.681545 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:31.697458 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1077/2167] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:38:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:38:31.704021 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87af5373-696d-4826-946a-f30705d81359 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:31.704317 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87af5373-696d-4826-946a-f30705d81359 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:31.705158 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87af5373-696d-4826-946a-f30705d81359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:31.726785 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-87af5373-696d-4826-946a-f30705d81359 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:38:31.728088 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-87af5373-696d-4826-946a-f30705d81359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:31.736171 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1091/2168] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:38:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:38:31.740440 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81532e21-e978-4169-86fb-5f96c0667c8a None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:31.752647 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81532e21-e978-4169-86fb-5f96c0667c8a None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:31.757785 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1747ec67-9d3d-4eef-b263-76c989be38ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:31.758526 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1747ec67-9d3d-4eef-b263-76c989be38ea admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:31.758886 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1747ec67-9d3d-4eef-b263-76c989be38ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:31.759226 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1747ec67-9d3d-4eef-b263-76c989be38ea admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:31.760124 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1747ec67-9d3d-4eef-b263-76c989be38ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:38:31.760504 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1747ec67-9d3d-4eef-b263-76c989be38ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:31.762192 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1747ec67-9d3d-4eef-b263-76c989be38ea admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:31.778871 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1078/2169] 10.222.0.99 () {58 vars in 1152 bytes} [Thu Jul 14 15:38:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:38:31.784282 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87af5373-696d-4826-946a-f30705d81359 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:31.793946 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87af5373-696d-4826-946a-f30705d81359 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:31.798687 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba70da8c-a49e-47f1-9df9-55de9b194ece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:31.799719 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba70da8c-a49e-47f1-9df9-55de9b194ece admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:31.799854 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba70da8c-a49e-47f1-9df9-55de9b194ece admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:31.800147 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba70da8c-a49e-47f1-9df9-55de9b194ece admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:31.800849 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba70da8c-a49e-47f1-9df9-55de9b194ece admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:31.802787 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba70da8c-a49e-47f1-9df9-55de9b194ece admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:31.834938 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1092/2170] 10.222.0.99 () {60 vars in 1152 bytes} [Thu Jul 14 15:38:31 2022] POST /identity/v3/projects => generated 391 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:38:31.835851 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1747ec67-9d3d-4eef-b263-76c989be38ea admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:31.847004 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1747ec67-9d3d-4eef-b263-76c989be38ea admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:31.907040 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14ef00d-7564-464d-9d33-f02ac124d88d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:31.907829 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14ef00d-7564-464d-9d33-f02ac124d88d admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:31.908120 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14ef00d-7564-464d-9d33-f02ac124d88d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:31.908585 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14ef00d-7564-464d-9d33-f02ac124d88d admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:31.909296 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14ef00d-7564-464d-9d33-f02ac124d88d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:31.911272 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14ef00d-7564-464d-9d33-f02ac124d88d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:31.931966 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1079/2171] 10.222.0.99 () {60 vars in 1143 bytes} [Thu Jul 14 15:38:31 2022] POST /identity/v3/users => generated 296 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 15:38:31.936152 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba70da8c-a49e-47f1-9df9-55de9b194ece admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:31.946844 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba70da8c-a49e-47f1-9df9-55de9b194ece admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:31.951401 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d124b906-7ed7-4b49-b007-cce30ed3d741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:31.952085 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d124b906-7ed7-4b49-b007-cce30ed3d741 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:31.952372 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d124b906-7ed7-4b49-b007-cce30ed3d741 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:31.952649 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d124b906-7ed7-4b49-b007-cce30ed3d741 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:31.953429 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d124b906-7ed7-4b49-b007-cce30ed3d741 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:31.956282 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d124b906-7ed7-4b49-b007-cce30ed3d741 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:31.966242 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1093/2172] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:38:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:38:31.970430 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14ef00d-7564-464d-9d33-f02ac124d88d admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:31.981238 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14ef00d-7564-464d-9d33-f02ac124d88d admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:31.987407 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb67cdd4-b9f5-4fa2-afc6-7dc6215b0ff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:31.988127 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb67cdd4-b9f5-4fa2-afc6-7dc6215b0ff7 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:31.988419 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb67cdd4-b9f5-4fa2-afc6-7dc6215b0ff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:31.988706 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb67cdd4-b9f5-4fa2-afc6-7dc6215b0ff7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:31.989456 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb67cdd4-b9f5-4fa2-afc6-7dc6215b0ff7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:31.991082 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb67cdd4-b9f5-4fa2-afc6-7dc6215b0ff7 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:32.005019 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1080/2173] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:38:31 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:38:32.009190 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d124b906-7ed7-4b49-b007-cce30ed3d741 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:32.019926 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d124b906-7ed7-4b49-b007-cce30ed3d741 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:32.024686 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a48632e-71bf-41ab-a46e-5a0014dbd375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:32.025560 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a48632e-71bf-41ab-a46e-5a0014dbd375 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:32.025843 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a48632e-71bf-41ab-a46e-5a0014dbd375 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:32.026117 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a48632e-71bf-41ab-a46e-5a0014dbd375 admin admin] PATH_INFO: `/v3/projects/af92057e97a842b1927404945426e225/users/64d8bab0d3f746a2b5d1f16a26a32f02/roles/b8c98fddc0fd4436b577fd1d3361bc38` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:32.041060 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a48632e-71bf-41ab-a46e-5a0014dbd375 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af92057e97a842b1927404945426e225, user_id=64d8bab0d3f746a2b5d1f16a26a32f02, role_id=b8c98fddc0fd4436b577fd1d3361bc38)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:32.043577 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a48632e-71bf-41ab-a46e-5a0014dbd375 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:32.053196 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:32.060394 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1094/2174] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 15:38:32 2022] PUT /identity/v3/projects/af92057e97a842b1927404945426e225/users/64d8bab0d3f746a2b5d1f16a26a32f02/roles/b8c98fddc0fd4436b577fd1d3361bc38 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:38:32.071414 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2c54fe-cf87-4a4e-a4b4-72187858f466 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:32.071902 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2c54fe-cf87-4a4e-a4b4-72187858f466 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:32.072040 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2c54fe-cf87-4a4e-a4b4-72187858f466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:32.099578 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-fc2c54fe-cf87-4a4e-a4b4-72187858f466 None None] MFA Rules not processed for user `64d8bab0d3f746a2b5d1f16a26a32f02`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:38:32.100858 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-fc2c54fe-cf87-4a4e-a4b4-72187858f466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:32.181782 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1081/2175] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:38:32 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:38:32.186414 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a48632e-71bf-41ab-a46e-5a0014dbd375 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:32.255625 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a48632e-71bf-41ab-a46e-5a0014dbd375 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:32.260529 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecbb283-c3ac-4b88-8ca9-8e262c5454af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:32.261193 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecbb283-c3ac-4b88-8ca9-8e262c5454af admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:32.261504 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecbb283-c3ac-4b88-8ca9-8e262c5454af admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:32.261780 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecbb283-c3ac-4b88-8ca9-8e262c5454af admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:32.262479 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecbb283-c3ac-4b88-8ca9-8e262c5454af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:32.264419 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecbb283-c3ac-4b88-8ca9-8e262c5454af admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:32.295189 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1095/2176] 10.222.0.99 () {60 vars in 1152 bytes} [Thu Jul 14 15:38:32 2022] POST /identity/v3/projects => generated 391 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 15:38:32.296295 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2c54fe-cf87-4a4e-a4b4-72187858f466 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:32.307208 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2c54fe-cf87-4a4e-a4b4-72187858f466 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:32.356598 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:32.367376 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832bfc92-6e01-4fd7-85ac-6f77c1399000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:32.368158 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832bfc92-6e01-4fd7-85ac-6f77c1399000 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:32.368473 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832bfc92-6e01-4fd7-85ac-6f77c1399000 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:32.368758 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832bfc92-6e01-4fd7-85ac-6f77c1399000 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:32.369457 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832bfc92-6e01-4fd7-85ac-6f77c1399000 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:32.371476 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832bfc92-6e01-4fd7-85ac-6f77c1399000 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:32.391671 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1082/2177] 10.222.0.99 () {60 vars in 1143 bytes} [Thu Jul 14 15:38:32 2022] POST /identity/v3/users => generated 302 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 15:38:32.396300 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecbb283-c3ac-4b88-8ca9-8e262c5454af admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:32.407065 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecbb283-c3ac-4b88-8ca9-8e262c5454af admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:32.412881 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bced8e6e-e4cc-4083-a241-909f9735d8b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:32.413556 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bced8e6e-e4cc-4083-a241-909f9735d8b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:32.413848 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bced8e6e-e4cc-4083-a241-909f9735d8b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:32.414133 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bced8e6e-e4cc-4083-a241-909f9735d8b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:32.414876 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bced8e6e-e4cc-4083-a241-909f9735d8b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:32.416525 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bced8e6e-e4cc-4083-a241-909f9735d8b7 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:32.427577 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1096/2178] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:38:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:38:32.432014 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832bfc92-6e01-4fd7-85ac-6f77c1399000 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:32.442648 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832bfc92-6e01-4fd7-85ac-6f77c1399000 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:32.447730 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90ed287-8d1c-4db4-ba28-1e05947992b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:32.448604 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90ed287-8d1c-4db4-ba28-1e05947992b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:32.449116 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90ed287-8d1c-4db4-ba28-1e05947992b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:32.449476 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90ed287-8d1c-4db4-ba28-1e05947992b4 admin admin] PATH_INFO: `/v3/projects/f90b238b4fba40e9987a7defd7d4d0a9/users/aeed3ace868d44e7af0004a89623fbdd/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:32.467471 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90ed287-8d1c-4db4-ba28-1e05947992b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f90b238b4fba40e9987a7defd7d4d0a9, user_id=aeed3ace868d44e7af0004a89623fbdd, role_id=d0f6ba1314f14d9f9c816bdb5d0c3aac)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:32.470950 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90ed287-8d1c-4db4-ba28-1e05947992b4 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:32.484092 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1083/2179] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 15:38:32 2022] PUT /identity/v3/projects/f90b238b4fba40e9987a7defd7d4d0a9/users/aeed3ace868d44e7af0004a89623fbdd/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 15:38:32.488344 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bced8e6e-e4cc-4083-a241-909f9735d8b7 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:32.499239 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bced8e6e-e4cc-4083-a241-909f9735d8b7 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:32.557755 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e907c3b-38bd-47cf-b48c-eb14ba82a80e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:32.558373 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e907c3b-38bd-47cf-b48c-eb14ba82a80e admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:32.558664 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e907c3b-38bd-47cf-b48c-eb14ba82a80e admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:32.558948 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e907c3b-38bd-47cf-b48c-eb14ba82a80e admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:32.559717 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e907c3b-38bd-47cf-b48c-eb14ba82a80e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:32.561494 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e907c3b-38bd-47cf-b48c-eb14ba82a80e admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:32.574195 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1097/2180] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:38:32 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:38:32.578376 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90ed287-8d1c-4db4-ba28-1e05947992b4 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:32.589495 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90ed287-8d1c-4db4-ba28-1e05947992b4 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:32.594205 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e85613-920c-4db1-8baf-30132f261520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:32.595085 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e85613-920c-4db1-8baf-30132f261520 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:32.596535 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e85613-920c-4db1-8baf-30132f261520 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:32.596818 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e85613-920c-4db1-8baf-30132f261520 admin admin] PATH_INFO: `/v3/projects/f90b238b4fba40e9987a7defd7d4d0a9/users/aeed3ace868d44e7af0004a89623fbdd/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:32.598820 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e85613-920c-4db1-8baf-30132f261520 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f90b238b4fba40e9987a7defd7d4d0a9, user_id=aeed3ace868d44e7af0004a89623fbdd, role_id=d0f6ba1314f14d9f9c816bdb5d0c3aac)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:32.601513 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e85613-920c-4db1-8baf-30132f261520 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:32.612541 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1084/2181] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 15:38:32 2022] PUT /identity/v3/projects/f90b238b4fba40e9987a7defd7d4d0a9/users/aeed3ace868d44e7af0004a89623fbdd/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 15:38:32.624178 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11884910-c860-4b03-a841-50d70846c03a None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:32.624507 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11884910-c860-4b03-a841-50d70846c03a None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:32.624806 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11884910-c860-4b03-a841-50d70846c03a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:32.653440 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-11884910-c860-4b03-a841-50d70846c03a None None] MFA Rules not processed for user `aeed3ace868d44e7af0004a89623fbdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:38:32.654701 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-11884910-c860-4b03-a841-50d70846c03a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:32.716522 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:32.729575 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1098/2182] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:38:32 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:38:32.741646 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e85613-920c-4db1-8baf-30132f261520 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:32.743482 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-46e85613-920c-4db1-8baf-30132f261520 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:32.810222 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e85613-920c-4db1-8baf-30132f261520 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:32.814631 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42217ea-f7ab-4e4f-9363-a66bbb1f6d06 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3e1a2b83f3d4d88ba5084dfaffd22f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:32.815377 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42217ea-f7ab-4e4f-9363-a66bbb1f6d06 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:32.815707 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42217ea-f7ab-4e4f-9363-a66bbb1f6d06 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:32.815992 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42217ea-f7ab-4e4f-9363-a66bbb1f6d06 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:32.824985 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42217ea-f7ab-4e4f-9363-a66bbb1f6d06 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:32.826487 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42217ea-f7ab-4e4f-9363-a66bbb1f6d06 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:32.837141 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1085/2183] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:38:32 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:38:32.884786 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11884910-c860-4b03-a841-50d70846c03a None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:32.886252 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-11884910-c860-4b03-a841-50d70846c03a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:32.900153 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11884910-c860-4b03-a841-50d70846c03a None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:32.903809 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6159d65f-399c-4735-a127-511384ac4900 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3e1a2b83f3d4d88ba5084dfaffd22f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:32.904465 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6159d65f-399c-4735-a127-511384ac4900 service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:32.904754 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6159d65f-399c-4735-a127-511384ac4900 service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:32.905035 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6159d65f-399c-4735-a127-511384ac4900 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:32.965858 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6159d65f-399c-4735-a127-511384ac4900 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:32.967325 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6159d65f-399c-4735-a127-511384ac4900 service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:32.977495 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1099/2184] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:38:32 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:38:35.983262 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42217ea-f7ab-4e4f-9363-a66bbb1f6d06 service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:35.984894 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d42217ea-f7ab-4e4f-9363-a66bbb1f6d06 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:36.053910 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42217ea-f7ab-4e4f-9363-a66bbb1f6d06 service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:36.058571 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da4ff04-4178-4338-8591-1bf5c97bd418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:36.059445 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da4ff04-4178-4338-8591-1bf5c97bd418 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:36.059768 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da4ff04-4178-4338-8591-1bf5c97bd418 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:36.060062 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da4ff04-4178-4338-8591-1bf5c97bd418 admin admin] PATH_INFO: `/v3/users/d1456f1f29f54585882a2a4872287e8a` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:36.061283 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da4ff04-4178-4338-8591-1bf5c97bd418 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1456f1f29f54585882a2a4872287e8a)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:36.063242 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da4ff04-4178-4338-8591-1bf5c97bd418 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:36.116720 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-7da4ff04-4178-4338-8591-1bf5c97bd418 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1456f1f29f54585882a2a4872287e8a'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:36.118434 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:36.124960 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-7da4ff04-4178-4338-8591-1bf5c97bd418 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1456f1f29f54585882a2a4872287e8a'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:36.140366 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-7da4ff04-4178-4338-8591-1bf5c97bd418 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1456f1f29f54585882a2a4872287e8a'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:36.141085 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-7da4ff04-4178-4338-8591-1bf5c97bd418 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1456f1f29f54585882a2a4872287e8a'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:36.141710 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-7da4ff04-4178-4338-8591-1bf5c97bd418 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1456f1f29f54585882a2a4872287e8a'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:36.158457 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1086/2185] 10.222.0.99 () {58 vars in 1223 bytes} [Thu Jul 14 15:38:35 2022] DELETE /identity/v3/users/d1456f1f29f54585882a2a4872287e8a => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 15:38:36.163450 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787199bd-a746-4574-9472-2f4e0425699d None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:36.164052 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787199bd-a746-4574-9472-2f4e0425699d None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:36.164174 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787199bd-a746-4574-9472-2f4e0425699d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:36.182617 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-787199bd-a746-4574-9472-2f4e0425699d None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:38:36.184294 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-787199bd-a746-4574-9472-2f4e0425699d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:36.256512 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1100/2186] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:38:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:38:36.267898 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da4ff04-4178-4338-8591-1bf5c97bd418 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:36.269436 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-7da4ff04-4178-4338-8591-1bf5c97bd418 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:36.299735 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787199bd-a746-4574-9472-2f4e0425699d None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:36.300735 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-787199bd-a746-4574-9472-2f4e0425699d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:36.312802 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787199bd-a746-4574-9472-2f4e0425699d None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:36.317780 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dc6562-fd72-41b8-b039-599cab838332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:36.318664 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dc6562-fd72-41b8-b039-599cab838332 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:36.318957 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dc6562-fd72-41b8-b039-599cab838332 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:36.319243 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dc6562-fd72-41b8-b039-599cab838332 admin admin] PATH_INFO: `/v3/users/64d8bab0d3f746a2b5d1f16a26a32f02` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:36.320509 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dc6562-fd72-41b8-b039-599cab838332 admin admin] RBAC: Authorizing `identity:delete_user(user_id=64d8bab0d3f746a2b5d1f16a26a32f02)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:36.322462 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dc6562-fd72-41b8-b039-599cab838332 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:36.331977 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da4ff04-4178-4338-8591-1bf5c97bd418 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:36.335552 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11d9321-31fc-4a33-a4f8-5dee00a6630e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:36.336228 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11d9321-31fc-4a33-a4f8-5dee00a6630e service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:36.336515 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11d9321-31fc-4a33-a4f8-5dee00a6630e service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:36.336794 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11d9321-31fc-4a33-a4f8-5dee00a6630e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:36.346976 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11d9321-31fc-4a33-a4f8-5dee00a6630e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:36.348455 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11d9321-31fc-4a33-a4f8-5dee00a6630e service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:36.360996 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1087/2187] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:38:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 15:38:36.375580 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-59dc6562-fd72-41b8-b039-599cab838332 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64d8bab0d3f746a2b5d1f16a26a32f02'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:36.385404 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-59dc6562-fd72-41b8-b039-599cab838332 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64d8bab0d3f746a2b5d1f16a26a32f02'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:36.408607 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-59dc6562-fd72-41b8-b039-599cab838332 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64d8bab0d3f746a2b5d1f16a26a32f02'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:36.409674 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-59dc6562-fd72-41b8-b039-599cab838332 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64d8bab0d3f746a2b5d1f16a26a32f02'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:36.410522 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-59dc6562-fd72-41b8-b039-599cab838332 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64d8bab0d3f746a2b5d1f16a26a32f02'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:36.433748 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1101/2188] 10.222.0.99 () {58 vars in 1223 bytes} [Thu Jul 14 15:38:36 2022] DELETE /identity/v3/users/64d8bab0d3f746a2b5d1f16a26a32f02 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 15:38:36.435770 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf25193-3e9d-4937-8d04-465a47e2cf9e None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:36.436302 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf25193-3e9d-4937-8d04-465a47e2cf9e None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:36.436685 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf25193-3e9d-4937-8d04-465a47e2cf9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:36.462166 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-0bf25193-3e9d-4937-8d04-465a47e2cf9e None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:38:36.463395 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0bf25193-3e9d-4937-8d04-465a47e2cf9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:36.543609 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1088/2189] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:38:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 15:38:36.549857 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dc6562-fd72-41b8-b039-599cab838332 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:36.551676 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-59dc6562-fd72-41b8-b039-599cab838332 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:36.565145 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:36.659533 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dc6562-fd72-41b8-b039-599cab838332 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:36.663918 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74b8ffc-45df-4a10-bcc0-b5f9881f07cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:36.664753 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74b8ffc-45df-4a10-bcc0-b5f9881f07cb service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:36.665119 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74b8ffc-45df-4a10-bcc0-b5f9881f07cb service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:36.665777 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74b8ffc-45df-4a10-bcc0-b5f9881f07cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:36.682789 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74b8ffc-45df-4a10-bcc0-b5f9881f07cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:36.684384 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74b8ffc-45df-4a10-bcc0-b5f9881f07cb service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:36.700484 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1102/2190] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:38:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:38:36.731393 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf25193-3e9d-4937-8d04-465a47e2cf9e None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:36.733187 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0bf25193-3e9d-4937-8d04-465a47e2cf9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:36.753658 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf25193-3e9d-4937-8d04-465a47e2cf9e None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:36.759140 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d18d597-ab54-49f2-b5da-731806302390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:36.760029 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d18d597-ab54-49f2-b5da-731806302390 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:36.760318 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d18d597-ab54-49f2-b5da-731806302390 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:36.760601 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d18d597-ab54-49f2-b5da-731806302390 admin admin] PATH_INFO: `/v3/projects/22afea9c5f524cb8b7de7790eeb4a220` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:36.762707 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d18d597-ab54-49f2-b5da-731806302390 admin admin] RBAC: Authorizing `identity:delete_project(project_id=22afea9c5f524cb8b7de7790eeb4a220)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:36.764667 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d18d597-ab54-49f2-b5da-731806302390 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:36.820737 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-3d18d597-ab54-49f2-b5da-731806302390 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22afea9c5f524cb8b7de7790eeb4a220', 'request_id': 'req-3d18d597-ab54-49f2-b5da-731806302390'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:36.827853 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-3d18d597-ab54-49f2-b5da-731806302390 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22afea9c5f524cb8b7de7790eeb4a220', 'request_id': 'req-3d18d597-ab54-49f2-b5da-731806302390'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:36.835384 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-3d18d597-ab54-49f2-b5da-731806302390 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22afea9c5f524cb8b7de7790eeb4a220', 'request_id': 'req-3d18d597-ab54-49f2-b5da-731806302390'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:36.837226 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:INSERT=1,keystone:DELETE=9,keystone:SELECT=90 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:36.851501 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-3d18d597-ab54-49f2-b5da-731806302390 admin admin] The token cache is being invalidate because project 22afea9c5f524cb8b7de7790eeb4a220 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 15:38:36.851829 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-3d18d597-ab54-49f2-b5da-731806302390 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:36.852787 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-3d18d597-ab54-49f2-b5da-731806302390 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:36.854038 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1089/2191] 10.222.0.99 () {58 vars in 1232 bytes} [Thu Jul 14 15:38:36 2022] DELETE /identity/v3/projects/22afea9c5f524cb8b7de7790eeb4a220 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:38:36.858893 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611936c9-0e9f-4dc3-8a12-c0ee7dfeee27 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:36.859201 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611936c9-0e9f-4dc3-8a12-c0ee7dfeee27 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:36.859477 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611936c9-0e9f-4dc3-8a12-c0ee7dfeee27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:36.882684 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-611936c9-0e9f-4dc3-8a12-c0ee7dfeee27 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:38:36.884271 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-611936c9-0e9f-4dc3-8a12-c0ee7dfeee27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:36.956146 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1103/2192] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:38:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:38:36.964410 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa3249f-3e79-48ef-b728-922fb2ccd58b None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:36.964709 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa3249f-3e79-48ef-b728-922fb2ccd58b None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:36.964995 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa3249f-3e79-48ef-b728-922fb2ccd58b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:36.989324 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-eaa3249f-3e79-48ef-b728-922fb2ccd58b None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:38:36.990643 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-eaa3249f-3e79-48ef-b728-922fb2ccd58b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:37.006753 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1090/2193] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:38:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:38:37.011630 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611936c9-0e9f-4dc3-8a12-c0ee7dfeee27 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:37.022131 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611936c9-0e9f-4dc3-8a12-c0ee7dfeee27 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:37.027669 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa3249f-3e79-48ef-b728-922fb2ccd58b None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:37.027806 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df452a58-570e-4605-9862-d7f8f0afae2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:37.028478 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df452a58-570e-4605-9862-d7f8f0afae2f admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:37.028785 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df452a58-570e-4605-9862-d7f8f0afae2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:37.029072 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df452a58-570e-4605-9862-d7f8f0afae2f admin admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:37.029814 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df452a58-570e-4605-9862-d7f8f0afae2f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:38:37.030129 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df452a58-570e-4605-9862-d7f8f0afae2f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:37.030446 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-eaa3249f-3e79-48ef-b728-922fb2ccd58b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:37.031771 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df452a58-570e-4605-9862-d7f8f0afae2f admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:37.045789 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1104/2194] 10.222.0.99 () {58 vars in 1152 bytes} [Thu Jul 14 15:38:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:38:37.050372 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df452a58-570e-4605-9862-d7f8f0afae2f admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:37.059813 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa3249f-3e79-48ef-b728-922fb2ccd58b None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:37.063345 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df452a58-570e-4605-9862-d7f8f0afae2f admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:37.066610 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde99c7c-95d6-4799-a55e-da2d3e7ff7d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:37.067488 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde99c7c-95d6-4799-a55e-da2d3e7ff7d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:37.067833 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde99c7c-95d6-4799-a55e-da2d3e7ff7d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:37.068123 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde99c7c-95d6-4799-a55e-da2d3e7ff7d1 admin admin] PATH_INFO: `/v3/projects/af92057e97a842b1927404945426e225` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:37.069707 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde99c7c-95d6-4799-a55e-da2d3e7ff7d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=af92057e97a842b1927404945426e225)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:37.069849 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6a7971-0cc0-4009-b6df-a93b1a8bb8af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:37.070394 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6a7971-0cc0-4009-b6df-a93b1a8bb8af admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:37.070661 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6a7971-0cc0-4009-b6df-a93b1a8bb8af admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:37.070935 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6a7971-0cc0-4009-b6df-a93b1a8bb8af admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:37.071656 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6a7971-0cc0-4009-b6df-a93b1a8bb8af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:37.072039 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde99c7c-95d6-4799-a55e-da2d3e7ff7d1 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:37.073584 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6a7971-0cc0-4009-b6df-a93b1a8bb8af admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:37.109457 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1105/2195] 10.222.0.99 () {60 vars in 1152 bytes} [Thu Jul 14 15:38:37 2022] POST /identity/v3/projects => generated 383 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:38:37.112871 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6a7971-0cc0-4009-b6df-a93b1a8bb8af admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:37.132048 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-fde99c7c-95d6-4799-a55e-da2d3e7ff7d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af92057e97a842b1927404945426e225', 'request_id': 'req-fde99c7c-95d6-4799-a55e-da2d3e7ff7d1'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:37.138193 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-fde99c7c-95d6-4799-a55e-da2d3e7ff7d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af92057e97a842b1927404945426e225', 'request_id': 'req-fde99c7c-95d6-4799-a55e-da2d3e7ff7d1'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:37.144221 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-fde99c7c-95d6-4799-a55e-da2d3e7ff7d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af92057e97a842b1927404945426e225', 'request_id': 'req-fde99c7c-95d6-4799-a55e-da2d3e7ff7d1'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:37.159329 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-fde99c7c-95d6-4799-a55e-da2d3e7ff7d1 admin admin] The token cache is being invalidate because project af92057e97a842b1927404945426e225 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 15:38:37.159837 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-fde99c7c-95d6-4799-a55e-da2d3e7ff7d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:37.161298 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-fde99c7c-95d6-4799-a55e-da2d3e7ff7d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:37.165198 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1091/2196] 10.222.0.99 () {58 vars in 1232 bytes} [Thu Jul 14 15:38:37 2022] DELETE /identity/v3/projects/af92057e97a842b1927404945426e225 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 15:38:37.170092 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde99c7c-95d6-4799-a55e-da2d3e7ff7d1 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:37.174258 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-fde99c7c-95d6-4799-a55e-da2d3e7ff7d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:37.185900 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6a7971-0cc0-4009-b6df-a93b1a8bb8af admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:37.204006 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:37.244631 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde99c7c-95d6-4799-a55e-da2d3e7ff7d1 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:37.248981 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cbddb1-2ef9-463f-924d-6de4b3ad82e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:37.249666 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cbddb1-2ef9-463f-924d-6de4b3ad82e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:37.249961 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cbddb1-2ef9-463f-924d-6de4b3ad82e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:37.250155 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b701ba3-869f-4b34-b615-9649942735ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:37.250298 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cbddb1-2ef9-463f-924d-6de4b3ad82e8 admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:37.250915 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cbddb1-2ef9-463f-924d-6de4b3ad82e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:37.252049 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b701ba3-869f-4b34-b615-9649942735ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:37.252464 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b701ba3-869f-4b34-b615-9649942735ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:37.252938 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b701ba3-869f-4b34-b615-9649942735ec admin admin] PATH_INFO: `/v3/users/aeed3ace868d44e7af0004a89623fbdd` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:37.253389 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cbddb1-2ef9-463f-924d-6de4b3ad82e8 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:37.254230 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b701ba3-869f-4b34-b615-9649942735ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=aeed3ace868d44e7af0004a89623fbdd)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:37.256263 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b701ba3-869f-4b34-b615-9649942735ec admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:37.275744 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1106/2197] 10.222.0.99 () {60 vars in 1143 bytes} [Thu Jul 14 15:38:37 2022] POST /identity/v3/users => generated 292 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:38:37.279901 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cbddb1-2ef9-463f-924d-6de4b3ad82e8 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:37.281544 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-f2cbddb1-2ef9-463f-924d-6de4b3ad82e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:37.294737 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cbddb1-2ef9-463f-924d-6de4b3ad82e8 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:37.300109 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5eb1e1-7cbe-40ef-afc8-e55588e6fd04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:37.300792 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5eb1e1-7cbe-40ef-afc8-e55588e6fd04 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:37.301085 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5eb1e1-7cbe-40ef-afc8-e55588e6fd04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:37.301370 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5eb1e1-7cbe-40ef-afc8-e55588e6fd04 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:37.302121 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5eb1e1-7cbe-40ef-afc8-e55588e6fd04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:37.303783 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5eb1e1-7cbe-40ef-afc8-e55588e6fd04 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:37.310617 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-6b701ba3-869f-4b34-b615-9649942735ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aeed3ace868d44e7af0004a89623fbdd'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:37.314624 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1107/2198] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:38:37 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 15:38:37.318900 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5eb1e1-7cbe-40ef-afc8-e55588e6fd04 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:37.319640 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-6b701ba3-869f-4b34-b615-9649942735ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aeed3ace868d44e7af0004a89623fbdd'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:37.328804 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5eb1e1-7cbe-40ef-afc8-e55588e6fd04 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:37.332424 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:INSERT=3,keystone:DELETE=18,keystone:SELECT=79 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:37.333772 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-6b701ba3-869f-4b34-b615-9649942735ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aeed3ace868d44e7af0004a89623fbdd'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:37.334671 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c9b23e-5c9b-4b22-9599-bceda4fe8357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:37.334805 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-6b701ba3-869f-4b34-b615-9649942735ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aeed3ace868d44e7af0004a89623fbdd'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:37.335466 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c9b23e-5c9b-4b22-9599-bceda4fe8357 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:37.335801 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c9b23e-5c9b-4b22-9599-bceda4fe8357 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:37.335927 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-6b701ba3-869f-4b34-b615-9649942735ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aeed3ace868d44e7af0004a89623fbdd'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:37.336204 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c9b23e-5c9b-4b22-9599-bceda4fe8357 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:37.337070 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c9b23e-5c9b-4b22-9599-bceda4fe8357 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:37.338901 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c9b23e-5c9b-4b22-9599-bceda4fe8357 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:37.351334 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1108/2199] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:38:37 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 15:38:37.351653 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1092/2200] 10.222.0.99 () {58 vars in 1223 bytes} [Thu Jul 14 15:38:37 2022] DELETE /identity/v3/users/aeed3ace868d44e7af0004a89623fbdd => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 15:38:37.355851 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c9b23e-5c9b-4b22-9599-bceda4fe8357 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:37.357671 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-d4c9b23e-5c9b-4b22-9599-bceda4fe8357 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:37.371377 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c9b23e-5c9b-4b22-9599-bceda4fe8357 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:37.437091 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02796ea1-210c-41e1-beb4-677e5efb1783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:37.438003 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02796ea1-210c-41e1-beb4-677e5efb1783 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:37.438292 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02796ea1-210c-41e1-beb4-677e5efb1783 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:37.438577 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02796ea1-210c-41e1-beb4-677e5efb1783 admin admin] PATH_INFO: `/v3/projects/858bb49b6e1e48568e2055eacc1ee7e5/users/71bc270c6253426baa4f79638d466b20/roles/b8c98fddc0fd4436b577fd1d3361bc38` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:37.455480 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02796ea1-210c-41e1-beb4-677e5efb1783 admin admin] RBAC: Authorizing `identity:create_grant(project_id=858bb49b6e1e48568e2055eacc1ee7e5, user_id=71bc270c6253426baa4f79638d466b20, role_id=b8c98fddc0fd4436b577fd1d3361bc38)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:37.458324 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02796ea1-210c-41e1-beb4-677e5efb1783 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:37.479684 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1109/2201] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 15:38:37 2022] PUT /identity/v3/projects/858bb49b6e1e48568e2055eacc1ee7e5/users/71bc270c6253426baa4f79638d466b20/roles/b8c98fddc0fd4436b577fd1d3361bc38 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 15:38:37.482917 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8959b068-fd62-463e-8461-0ad1ec4fc857 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:37.483223 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8959b068-fd62-463e-8461-0ad1ec4fc857 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:37.483509 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8959b068-fd62-463e-8461-0ad1ec4fc857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:37.504017 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-8959b068-fd62-463e-8461-0ad1ec4fc857 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:38:37.505220 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-8959b068-fd62-463e-8461-0ad1ec4fc857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:37.589135 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1093/2202] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:38:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:38:37.599131 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02796ea1-210c-41e1-beb4-677e5efb1783 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:37.600952 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-02796ea1-210c-41e1-beb4-677e5efb1783 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:37.656207 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8959b068-fd62-463e-8461-0ad1ec4fc857 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:37.658203 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-8959b068-fd62-463e-8461-0ad1ec4fc857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:37.669759 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02796ea1-210c-41e1-beb4-677e5efb1783 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:37.674683 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd9137d-2750-412b-b7a7-4f4505425602 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:37.675525 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd9137d-2750-412b-b7a7-4f4505425602 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:37.675858 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd9137d-2750-412b-b7a7-4f4505425602 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:37.676168 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd9137d-2750-412b-b7a7-4f4505425602 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:37.685784 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8959b068-fd62-463e-8461-0ad1ec4fc857 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:37.688559 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd9137d-2750-412b-b7a7-4f4505425602 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:37.690600 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd9137d-2750-412b-b7a7-4f4505425602 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:37.692157 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f7c0e5-077a-47c6-8dac-0f49291dacd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:37.693099 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f7c0e5-077a-47c6-8dac-0f49291dacd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:37.693380 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f7c0e5-077a-47c6-8dac-0f49291dacd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:37.693704 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f7c0e5-077a-47c6-8dac-0f49291dacd9 admin admin] PATH_INFO: `/v3/projects/f90b238b4fba40e9987a7defd7d4d0a9` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:37.694933 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f7c0e5-077a-47c6-8dac-0f49291dacd9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f90b238b4fba40e9987a7defd7d4d0a9)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:37.697259 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f7c0e5-077a-47c6-8dac-0f49291dacd9 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:37.700706 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1110/2203] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:38:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:38:37.755799 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-31f7c0e5-077a-47c6-8dac-0f49291dacd9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f90b238b4fba40e9987a7defd7d4d0a9', 'request_id': 'req-31f7c0e5-077a-47c6-8dac-0f49291dacd9'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:37.767742 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-31f7c0e5-077a-47c6-8dac-0f49291dacd9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f90b238b4fba40e9987a7defd7d4d0a9', 'request_id': 'req-31f7c0e5-077a-47c6-8dac-0f49291dacd9'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:37.779026 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-31f7c0e5-077a-47c6-8dac-0f49291dacd9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f90b238b4fba40e9987a7defd7d4d0a9', 'request_id': 'req-31f7c0e5-077a-47c6-8dac-0f49291dacd9'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:37.810775 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-31f7c0e5-077a-47c6-8dac-0f49291dacd9 admin admin] The token cache is being invalidate because project f90b238b4fba40e9987a7defd7d4d0a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 15:38:37.810775 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-31f7c0e5-077a-47c6-8dac-0f49291dacd9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:37.810775 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-31f7c0e5-077a-47c6-8dac-0f49291dacd9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:37.812161 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1094/2204] 10.222.0.99 () {58 vars in 1232 bytes} [Thu Jul 14 15:38:37 2022] DELETE /identity/v3/projects/f90b238b4fba40e9987a7defd7d4d0a9 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 15:38:37.817810 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7288852-fd3c-4d8c-9616-927023507587 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:37.817810 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7288852-fd3c-4d8c-9616-927023507587 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:37.818155 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7288852-fd3c-4d8c-9616-927023507587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:37.837987 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-b7288852-fd3c-4d8c-9616-927023507587 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:38:37.839385 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-b7288852-fd3c-4d8c-9616-927023507587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:37.850353 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:37.899248 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f7c0e5-077a-47c6-8dac-0f49291dacd9 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:37.902684 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-31f7c0e5-077a-47c6-8dac-0f49291dacd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:37.920219 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1111/2205] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:38:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:38:37.928918 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5b7607-e9eb-4184-8b3b-b4b7280aa313 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:37.929222 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5b7607-e9eb-4184-8b3b-b4b7280aa313 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:37.929512 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5b7607-e9eb-4184-8b3b-b4b7280aa313 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:37.950139 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-2a5b7607-e9eb-4184-8b3b-b4b7280aa313 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:38:37.951370 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-2a5b7607-e9eb-4184-8b3b-b4b7280aa313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:37.962758 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1112/2206] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:38:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:38:37.967558 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5b7607-e9eb-4184-8b3b-b4b7280aa313 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:37.978699 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5b7607-e9eb-4184-8b3b-b4b7280aa313 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:37.983660 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae1aa98-8abb-40bf-8d82-3d2e66073522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:37.984339 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae1aa98-8abb-40bf-8d82-3d2e66073522 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:37.984764 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae1aa98-8abb-40bf-8d82-3d2e66073522 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:37.985057 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae1aa98-8abb-40bf-8d82-3d2e66073522 admin admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:37.986436 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae1aa98-8abb-40bf-8d82-3d2e66073522 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:38:37.986832 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae1aa98-8abb-40bf-8d82-3d2e66073522 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:37.988656 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae1aa98-8abb-40bf-8d82-3d2e66073522 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:38.002976 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f7c0e5-077a-47c6-8dac-0f49291dacd9 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:38.007808 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1113/2207] 10.222.0.99 () {58 vars in 1152 bytes} [Thu Jul 14 15:38:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:38:38.009098 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae1aa98-8abb-40bf-8d82-3d2e66073522 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:38.009868 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e80abe-1f0a-4849-973e-896eaee794f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:38.011407 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e80abe-1f0a-4849-973e-896eaee794f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:38.012048 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e80abe-1f0a-4849-973e-896eaee794f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:38.012809 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e80abe-1f0a-4849-973e-896eaee794f8 admin admin] PATH_INFO: `/v3/users/e34e7260a8e8454ab282db7e8a50169e` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:38.014243 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e80abe-1f0a-4849-973e-896eaee794f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e34e7260a8e8454ab282db7e8a50169e)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:38.018587 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e80abe-1f0a-4849-973e-896eaee794f8 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:38.023507 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae1aa98-8abb-40bf-8d82-3d2e66073522 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:38.030147 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560060e5-d6c6-407b-bc63-cc11e41102af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:38.031572 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560060e5-d6c6-407b-bc63-cc11e41102af admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:38.032138 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560060e5-d6c6-407b-bc63-cc11e41102af admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:38.033316 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560060e5-d6c6-407b-bc63-cc11e41102af admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:38.034499 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560060e5-d6c6-407b-bc63-cc11e41102af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:38.036730 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560060e5-d6c6-407b-bc63-cc11e41102af admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:38.044298 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:38.069412 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1114/2208] 10.222.0.99 () {60 vars in 1152 bytes} [Thu Jul 14 15:38:38 2022] POST /identity/v3/projects => generated 381 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:38:38.073675 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560060e5-d6c6-407b-bc63-cc11e41102af admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:38.100363 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d5e80abe-1f0a-4849-973e-896eaee794f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e34e7260a8e8454ab282db7e8a50169e'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:38.113002 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d5e80abe-1f0a-4849-973e-896eaee794f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e34e7260a8e8454ab282db7e8a50169e'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:38.155866 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d5e80abe-1f0a-4849-973e-896eaee794f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e34e7260a8e8454ab282db7e8a50169e'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:38.156803 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d5e80abe-1f0a-4849-973e-896eaee794f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e34e7260a8e8454ab282db7e8a50169e'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:38.157758 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d5e80abe-1f0a-4849-973e-896eaee794f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e34e7260a8e8454ab282db7e8a50169e'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:38.177761 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560060e5-d6c6-407b-bc63-cc11e41102af admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:38.185491 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1095/2209] 10.222.0.99 () {58 vars in 1223 bytes} [Thu Jul 14 15:38:37 2022] DELETE /identity/v3/users/e34e7260a8e8454ab282db7e8a50169e => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:38:38.186337 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ef7886-e9b1-4eb7-8e13-f41954ef7a51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:38.187042 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ef7886-e9b1-4eb7-8e13-f41954ef7a51 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:38.187960 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ef7886-e9b1-4eb7-8e13-f41954ef7a51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:38.188269 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ef7886-e9b1-4eb7-8e13-f41954ef7a51 admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:38.189797 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ef7886-e9b1-4eb7-8e13-f41954ef7a51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:38.190937 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2318c36f-5712-4f26-98b3-f78e6003e501 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:38.191407 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2318c36f-5712-4f26-98b3-f78e6003e501 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:38.191909 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2318c36f-5712-4f26-98b3-f78e6003e501 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:38.196440 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ef7886-e9b1-4eb7-8e13-f41954ef7a51 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:38.213714 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-2318c36f-5712-4f26-98b3-f78e6003e501 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:38:38.215895 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-2318c36f-5712-4f26-98b3-f78e6003e501 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:38.236408 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1115/2210] 10.222.0.99 () {60 vars in 1143 bytes} [Thu Jul 14 15:38:38 2022] POST /identity/v3/users => generated 291 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 15:38:38.241406 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ef7886-e9b1-4eb7-8e13-f41954ef7a51 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:38.247812 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-e7ef7886-e9b1-4eb7-8e13-f41954ef7a51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:38.269542 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ef7886-e9b1-4eb7-8e13-f41954ef7a51 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:38.306987 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1096/2211] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:38:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 15:38:38.334034 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2318c36f-5712-4f26-98b3-f78e6003e501 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:38.335953 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-2318c36f-5712-4f26-98b3-f78e6003e501 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:38.352609 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ddd4d4-4817-4dbf-a326-245769f942b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:38.353443 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ddd4d4-4817-4dbf-a326-245769f942b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:38.353871 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ddd4d4-4817-4dbf-a326-245769f942b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:38.354152 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ddd4d4-4817-4dbf-a326-245769f942b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:38.354902 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ddd4d4-4817-4dbf-a326-245769f942b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:38.356626 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ddd4d4-4817-4dbf-a326-245769f942b9 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:38.358879 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:38.368784 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1116/2212] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:38:38 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:38:38.372743 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ddd4d4-4817-4dbf-a326-245769f942b9 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:38.383319 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ddd4d4-4817-4dbf-a326-245769f942b9 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:38.388657 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a244fa3-f140-442d-b1eb-5c907b9b1855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:38.389329 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a244fa3-f140-442d-b1eb-5c907b9b1855 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:38.389614 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a244fa3-f140-442d-b1eb-5c907b9b1855 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:38.389891 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a244fa3-f140-442d-b1eb-5c907b9b1855 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:38.390631 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a244fa3-f140-442d-b1eb-5c907b9b1855 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:38.393048 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a244fa3-f140-442d-b1eb-5c907b9b1855 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:38.407115 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1117/2213] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:38:38 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:38:38.411536 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a244fa3-f140-442d-b1eb-5c907b9b1855 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:38.424171 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a244fa3-f140-442d-b1eb-5c907b9b1855 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:38.428337 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2318c36f-5712-4f26-98b3-f78e6003e501 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:38.430643 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb52483-4824-4595-af43-3073b8d3026c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:38.431619 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb52483-4824-4595-af43-3073b8d3026c admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:38.431997 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb52483-4824-4595-af43-3073b8d3026c admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:38.432111 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88025bdf-dc85-435b-a3dd-e1ed6c1876d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:38.432401 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb52483-4824-4595-af43-3073b8d3026c admin admin] PATH_INFO: `/v3/projects/a69112f898d74cdea01cf1bae7dd30cd/users/caa8148a3163453881214e833d56c043/roles/b8c98fddc0fd4436b577fd1d3361bc38` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:38.432742 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88025bdf-dc85-435b-a3dd-e1ed6c1876d5 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:38.433031 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88025bdf-dc85-435b-a3dd-e1ed6c1876d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:38.433316 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88025bdf-dc85-435b-a3dd-e1ed6c1876d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:38.446351 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88025bdf-dc85-435b-a3dd-e1ed6c1876d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:38.447745 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88025bdf-dc85-435b-a3dd-e1ed6c1876d5 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:38.452713 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb52483-4824-4595-af43-3073b8d3026c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a69112f898d74cdea01cf1bae7dd30cd, user_id=caa8148a3163453881214e833d56c043, role_id=b8c98fddc0fd4436b577fd1d3361bc38)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:38.455761 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb52483-4824-4595-af43-3073b8d3026c admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:38.460176 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1097/2214] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:38:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:38:38.473008 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1118/2215] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 15:38:38 2022] PUT /identity/v3/projects/a69112f898d74cdea01cf1bae7dd30cd/users/caa8148a3163453881214e833d56c043/roles/b8c98fddc0fd4436b577fd1d3361bc38 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 15:38:38.484427 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed318bce-6fdc-4156-aed4-3fc7c62bc32e None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:38.484757 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed318bce-6fdc-4156-aed4-3fc7c62bc32e None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:38.485097 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed318bce-6fdc-4156-aed4-3fc7c62bc32e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:38.514472 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-ed318bce-6fdc-4156-aed4-3fc7c62bc32e None None] MFA Rules not processed for user `caa8148a3163453881214e833d56c043`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:38:38.515886 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ed318bce-6fdc-4156-aed4-3fc7c62bc32e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:38.540892 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:38.625315 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1098/2216] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:38:38 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 15:38:38.630756 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb52483-4824-4595-af43-3073b8d3026c admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:38.644690 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7323d3e9-4543-4df7-a8b9-bd7f98b23789 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:38.645277 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7323d3e9-4543-4df7-a8b9-bd7f98b23789 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:38.645769 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7323d3e9-4543-4df7-a8b9-bd7f98b23789 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:38.679229 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-7323d3e9-4543-4df7-a8b9-bd7f98b23789 None None] MFA Rules not processed for user `71bc270c6253426baa4f79638d466b20`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:38:38.680547 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-7323d3e9-4543-4df7-a8b9-bd7f98b23789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:38.725695 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb52483-4824-4595-af43-3073b8d3026c admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:38.735866 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6d3d0e-6ff0-4b43-9e86-7ca837a5dafa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:38.736632 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6d3d0e-6ff0-4b43-9e86-7ca837a5dafa admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:38.736903 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6d3d0e-6ff0-4b43-9e86-7ca837a5dafa admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:38.737563 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6d3d0e-6ff0-4b43-9e86-7ca837a5dafa admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:38.738290 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6d3d0e-6ff0-4b43-9e86-7ca837a5dafa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:38.742585 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6d3d0e-6ff0-4b43-9e86-7ca837a5dafa admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:38.764534 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1099/2217] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:38:38 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 15:38:38.778377 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7323d3e9-4543-4df7-a8b9-bd7f98b23789 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:38.780709 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-7323d3e9-4543-4df7-a8b9-bd7f98b23789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:38.802203 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1119/2218] 10.222.0.99 () {60 vars in 1152 bytes} [Thu Jul 14 15:38:38 2022] POST /identity/v3/projects => generated 381 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:38:38.806948 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6d3d0e-6ff0-4b43-9e86-7ca837a5dafa admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:38.874910 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7323d3e9-4543-4df7-a8b9-bd7f98b23789 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:38.914564 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6d3d0e-6ff0-4b43-9e86-7ca837a5dafa admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:38.921380 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2efb23f-2421-472f-8a8d-a5b27422c5d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:38.922074 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2efb23f-2421-472f-8a8d-a5b27422c5d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:38.922399 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2efb23f-2421-472f-8a8d-a5b27422c5d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:38.922688 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2efb23f-2421-472f-8a8d-a5b27422c5d3 admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:38.923449 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2efb23f-2421-472f-8a8d-a5b27422c5d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:38.925561 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2efb23f-2421-472f-8a8d-a5b27422c5d3 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:38.934166 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fdebfb-7c4a-4e2c-a9fb-c29fb5c387ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3e1a2b83f3d4d88ba5084dfaffd22f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:38.934897 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fdebfb-7c4a-4e2c-a9fb-c29fb5c387ea service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:38.935325 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fdebfb-7c4a-4e2c-a9fb-c29fb5c387ea service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:38.935705 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fdebfb-7c4a-4e2c-a9fb-c29fb5c387ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:38.947656 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1120/2219] 10.222.0.99 () {60 vars in 1143 bytes} [Thu Jul 14 15:38:38 2022] POST /identity/v3/users => generated 297 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:38:38.951440 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2efb23f-2421-472f-8a8d-a5b27422c5d3 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:38.953400 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-f2efb23f-2421-472f-8a8d-a5b27422c5d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:38.959306 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:38.966523 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2efb23f-2421-472f-8a8d-a5b27422c5d3 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:38.971543 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4334fa5d-c4b7-4b28-ba27-bc50288f0735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:38.972441 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4334fa5d-c4b7-4b28-ba27-bc50288f0735 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:38.972737 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4334fa5d-c4b7-4b28-ba27-bc50288f0735 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:38.973030 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4334fa5d-c4b7-4b28-ba27-bc50288f0735 admin admin] PATH_INFO: `/v3/projects/df766904ce8c45fc9eb0b59e959f8583` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:38.974236 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4334fa5d-c4b7-4b28-ba27-bc50288f0735 admin admin] RBAC: Authorizing `identity:delete_project(project_id=df766904ce8c45fc9eb0b59e959f8583)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:38.976347 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4334fa5d-c4b7-4b28-ba27-bc50288f0735 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:39.002286 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fdebfb-7c4a-4e2c-a9fb-c29fb5c387ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:39.003811 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fdebfb-7c4a-4e2c-a9fb-c29fb5c387ea service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:39.013901 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1100/2220] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:38:38 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:38:39.014741 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fdebfb-7c4a-4e2c-a9fb-c29fb5c387ea service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:39.022826 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:39.026385 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fdebfb-7c4a-4e2c-a9fb-c29fb5c387ea service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:39.045847 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-4334fa5d-c4b7-4b28-ba27-bc50288f0735 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df766904ce8c45fc9eb0b59e959f8583', 'request_id': 'req-4334fa5d-c4b7-4b28-ba27-bc50288f0735'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:39.055691 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-4334fa5d-c4b7-4b28-ba27-bc50288f0735 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df766904ce8c45fc9eb0b59e959f8583', 'request_id': 'req-4334fa5d-c4b7-4b28-ba27-bc50288f0735'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:39.064544 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-4334fa5d-c4b7-4b28-ba27-bc50288f0735 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df766904ce8c45fc9eb0b59e959f8583', 'request_id': 'req-4334fa5d-c4b7-4b28-ba27-bc50288f0735'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:39.082599 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-4334fa5d-c4b7-4b28-ba27-bc50288f0735 admin admin] The token cache is being invalidate because project df766904ce8c45fc9eb0b59e959f8583 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 15:38:39.082888 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-4334fa5d-c4b7-4b28-ba27-bc50288f0735 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:39.083732 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-4334fa5d-c4b7-4b28-ba27-bc50288f0735 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:39.085693 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1121/2221] 10.222.0.99 () {58 vars in 1232 bytes} [Thu Jul 14 15:38:38 2022] DELETE /identity/v3/projects/df766904ce8c45fc9eb0b59e959f8583 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:38:39.092920 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc934c4-dc74-46c1-9cff-f91f6fc9819d None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:39.093284 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc934c4-dc74-46c1-9cff-f91f6fc9819d None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:39.093814 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc934c4-dc74-46c1-9cff-f91f6fc9819d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:39.097683 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0679059-79c6-4c86-a71e-fb377c759adc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:39.098607 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0679059-79c6-4c86-a71e-fb377c759adc admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:39.099052 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0679059-79c6-4c86-a71e-fb377c759adc admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:39.099538 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0679059-79c6-4c86-a71e-fb377c759adc admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:39.100518 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0679059-79c6-4c86-a71e-fb377c759adc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:39.102573 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0679059-79c6-4c86-a71e-fb377c759adc admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:39.121834 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1101/2222] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:38:39 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:38:39.122201 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-5fc934c4-dc74-46c1-9cff-f91f6fc9819d None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:38:39.122564 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0679059-79c6-4c86-a71e-fb377c759adc admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:39.123301 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-5fc934c4-dc74-46c1-9cff-f91f6fc9819d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:39.136412 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1122/2223] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:38:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:38:39.138031 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d0679059-79c6-4c86-a71e-fb377c759adc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:39.141374 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea14b6c7-b98c-4030-8578-207aa1b5885a None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:39.143377 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea14b6c7-b98c-4030-8578-207aa1b5885a None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:39.144021 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea14b6c7-b98c-4030-8578-207aa1b5885a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:39.153803 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0679059-79c6-4c86-a71e-fb377c759adc admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:39.161641 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75da12e3-de23-4ebf-8e86-9dc6d249f57d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:39.162566 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75da12e3-de23-4ebf-8e86-9dc6d249f57d admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:39.162858 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75da12e3-de23-4ebf-8e86-9dc6d249f57d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:39.163146 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75da12e3-de23-4ebf-8e86-9dc6d249f57d admin admin] PATH_INFO: `/v3/projects/d0f63223785242fba7905a51cd2c1d0c/users/0eefe94420b445b6adc9284165ea34c3/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:39.170558 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-ea14b6c7-b98c-4030-8578-207aa1b5885a None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:38:39.172085 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-ea14b6c7-b98c-4030-8578-207aa1b5885a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:39.182981 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1123/2224] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:38:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:38:39.184756 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75da12e3-de23-4ebf-8e86-9dc6d249f57d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0f63223785242fba7905a51cd2c1d0c, user_id=0eefe94420b445b6adc9284165ea34c3, role_id=d0f6ba1314f14d9f9c816bdb5d0c3aac)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:39.186916 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea14b6c7-b98c-4030-8578-207aa1b5885a None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:39.188488 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75da12e3-de23-4ebf-8e86-9dc6d249f57d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:39.200421 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea14b6c7-b98c-4030-8578-207aa1b5885a None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:39.209347 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1102/2225] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 15:38:39 2022] PUT /identity/v3/projects/d0f63223785242fba7905a51cd2c1d0c/users/0eefe94420b445b6adc9284165ea34c3/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:38:39.217662 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75da12e3-de23-4ebf-8e86-9dc6d249f57d admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:39.284071 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c6208e-a4c1-4554-a41f-cd624b1373af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:39.284793 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c6208e-a4c1-4554-a41f-cd624b1373af admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:39.285264 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c6208e-a4c1-4554-a41f-cd624b1373af admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:39.285617 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c6208e-a4c1-4554-a41f-cd624b1373af admin admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:39.286554 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c6208e-a4c1-4554-a41f-cd624b1373af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:38:39.286874 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c6208e-a4c1-4554-a41f-cd624b1373af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:39.288717 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c6208e-a4c1-4554-a41f-cd624b1373af admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:39.305655 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75da12e3-de23-4ebf-8e86-9dc6d249f57d admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:39.307559 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1124/2226] 10.222.0.99 () {58 vars in 1152 bytes} [Thu Jul 14 15:38:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:38:39.308558 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c6208e-a4c1-4554-a41f-cd624b1373af admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:39.310801 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de648c98-3f0a-40a8-881d-3a8ff6cb0448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:39.311487 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de648c98-3f0a-40a8-881d-3a8ff6cb0448 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:39.311816 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de648c98-3f0a-40a8-881d-3a8ff6cb0448 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:39.312109 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de648c98-3f0a-40a8-881d-3a8ff6cb0448 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:39.312858 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de648c98-3f0a-40a8-881d-3a8ff6cb0448 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:39.314861 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de648c98-3f0a-40a8-881d-3a8ff6cb0448 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:39.322491 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c6208e-a4c1-4554-a41f-cd624b1373af admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:39.327650 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cdc83e-b799-431d-a8f3-825c436665ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:39.328325 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cdc83e-b799-431d-a8f3-825c436665ce admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:39.328617 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cdc83e-b799-431d-a8f3-825c436665ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:39.328936 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cdc83e-b799-431d-a8f3-825c436665ce admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:39.330213 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cdc83e-b799-431d-a8f3-825c436665ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:39.332470 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cdc83e-b799-431d-a8f3-825c436665ce admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:39.336724 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1103/2227] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:38:39 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:38:39.341111 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de648c98-3f0a-40a8-881d-3a8ff6cb0448 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:39.354126 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de648c98-3f0a-40a8-881d-3a8ff6cb0448 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:39.360761 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471075f1-b258-4a56-8b5c-6aec849b3234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:39.362052 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471075f1-b258-4a56-8b5c-6aec849b3234 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:39.362410 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471075f1-b258-4a56-8b5c-6aec849b3234 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:39.362763 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471075f1-b258-4a56-8b5c-6aec849b3234 admin admin] PATH_INFO: `/v3/projects/d0f63223785242fba7905a51cd2c1d0c/users/0eefe94420b445b6adc9284165ea34c3/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:39.364908 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471075f1-b258-4a56-8b5c-6aec849b3234 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0f63223785242fba7905a51cd2c1d0c, user_id=0eefe94420b445b6adc9284165ea34c3, role_id=d0f6ba1314f14d9f9c816bdb5d0c3aac)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:39.367580 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471075f1-b258-4a56-8b5c-6aec849b3234 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:39.369488 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1125/2228] 10.222.0.99 () {60 vars in 1152 bytes} [Thu Jul 14 15:38:39 2022] POST /identity/v3/projects => generated 383 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 15:38:39.373310 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cdc83e-b799-431d-a8f3-825c436665ce admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:39.385432 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1104/2229] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 15:38:39 2022] PUT /identity/v3/projects/d0f63223785242fba7905a51cd2c1d0c/users/0eefe94420b445b6adc9284165ea34c3/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 15:38:39.398664 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effd83a4-816f-4a5a-9ea3-de2f3acc26fe None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:39.398967 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effd83a4-816f-4a5a-9ea3-de2f3acc26fe None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:39.399254 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effd83a4-816f-4a5a-9ea3-de2f3acc26fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:39.428763 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-effd83a4-816f-4a5a-9ea3-de2f3acc26fe None None] MFA Rules not processed for user `0eefe94420b445b6adc9284165ea34c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:38:39.430016 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-effd83a4-816f-4a5a-9ea3-de2f3acc26fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:39.435761 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:39.463233 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cdc83e-b799-431d-a8f3-825c436665ce admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:39.490737 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:39.511671 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1105/2230] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:38:39 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 15:38:39.524866 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effd83a4-816f-4a5a-9ea3-de2f3acc26fe None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:39.526270 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-effd83a4-816f-4a5a-9ea3-de2f3acc26fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:39.527476 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905160d3-5a27-4bb7-bb5d-5434eee814e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:39.528190 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905160d3-5a27-4bb7-bb5d-5434eee814e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:39.528480 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905160d3-5a27-4bb7-bb5d-5434eee814e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:39.528760 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905160d3-5a27-4bb7-bb5d-5434eee814e3 admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:39.529488 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905160d3-5a27-4bb7-bb5d-5434eee814e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:39.531533 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905160d3-5a27-4bb7-bb5d-5434eee814e3 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:39.552261 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1126/2231] 10.222.0.99 () {60 vars in 1143 bytes} [Thu Jul 14 15:38:39 2022] POST /identity/v3/users => generated 292 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:38:39.556390 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905160d3-5a27-4bb7-bb5d-5434eee814e3 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:39.568930 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905160d3-5a27-4bb7-bb5d-5434eee814e3 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:39.574037 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36775791-2509-4f0f-986a-4da036c901ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:39.574704 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36775791-2509-4f0f-986a-4da036c901ec admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:39.574999 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36775791-2509-4f0f-986a-4da036c901ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:39.575282 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36775791-2509-4f0f-986a-4da036c901ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:39.576178 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36775791-2509-4f0f-986a-4da036c901ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:39.577848 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36775791-2509-4f0f-986a-4da036c901ec admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:39.586939 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effd83a4-816f-4a5a-9ea3-de2f3acc26fe None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:39.588627 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1127/2232] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:38:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 15:38:39.590762 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506495e1-9e4d-406a-b75b-6110a1746996 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3e1a2b83f3d4d88ba5084dfaffd22f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:39.591414 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506495e1-9e4d-406a-b75b-6110a1746996 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:39.591737 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506495e1-9e4d-406a-b75b-6110a1746996 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:39.592022 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506495e1-9e4d-406a-b75b-6110a1746996 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:39.593401 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36775791-2509-4f0f-986a-4da036c901ec admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:39.602236 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506495e1-9e4d-406a-b75b-6110a1746996 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:39.603578 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36775791-2509-4f0f-986a-4da036c901ec admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:39.603711 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506495e1-9e4d-406a-b75b-6110a1746996 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:39.608324 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e83e137-364a-475b-b23d-5e73bac038a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:39.608978 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e83e137-364a-475b-b23d-5e73bac038a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:39.609268 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e83e137-364a-475b-b23d-5e73bac038a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:39.609549 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e83e137-364a-475b-b23d-5e73bac038a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:39.610286 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e83e137-364a-475b-b23d-5e73bac038a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:39.612150 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e83e137-364a-475b-b23d-5e73bac038a6 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:39.613837 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1106/2233] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:38:39 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 15:38:39.622561 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1128/2234] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:38:39 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:38:39.627056 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506495e1-9e4d-406a-b75b-6110a1746996 service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:39.641206 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506495e1-9e4d-406a-b75b-6110a1746996 service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:39.647460 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6f37ed-cb10-4696-acdc-91f13dae9625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:39.648431 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6f37ed-cb10-4696-acdc-91f13dae9625 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:39.648722 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6f37ed-cb10-4696-acdc-91f13dae9625 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:39.649026 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6f37ed-cb10-4696-acdc-91f13dae9625 admin admin] PATH_INFO: `/v3/projects/1d871080e80348f48bc10813e9649548/users/485f3a4cdf384a7e9de6e578f6dee765/roles/b8c98fddc0fd4436b577fd1d3361bc38` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:39.676373 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6f37ed-cb10-4696-acdc-91f13dae9625 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d871080e80348f48bc10813e9649548, user_id=485f3a4cdf384a7e9de6e578f6dee765, role_id=b8c98fddc0fd4436b577fd1d3361bc38)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:39.679410 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6f37ed-cb10-4696-acdc-91f13dae9625 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:39.697132 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1107/2235] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 15:38:39 2022] PUT /identity/v3/projects/1d871080e80348f48bc10813e9649548/users/485f3a4cdf384a7e9de6e578f6dee765/roles/b8c98fddc0fd4436b577fd1d3361bc38 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 15:38:39.708918 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a597d93a-f14b-4aca-b62c-fc1d62247223 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:39.709237 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a597d93a-f14b-4aca-b62c-fc1d62247223 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:39.709528 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a597d93a-f14b-4aca-b62c-fc1d62247223 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:39.738477 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-a597d93a-f14b-4aca-b62c-fc1d62247223 None None] MFA Rules not processed for user `485f3a4cdf384a7e9de6e578f6dee765`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:38:39.739617 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-a597d93a-f14b-4aca-b62c-fc1d62247223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:39.817407 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1129/2236] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:38:39 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:38:39.818368 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6f37ed-cb10-4696-acdc-91f13dae9625 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:39.884429 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6f37ed-cb10-4696-acdc-91f13dae9625 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:39.890569 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171b3a89-454a-473c-b7ad-984a3fbc6c74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:39.891287 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171b3a89-454a-473c-b7ad-984a3fbc6c74 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:39.891617 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171b3a89-454a-473c-b7ad-984a3fbc6c74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:39.892380 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171b3a89-454a-473c-b7ad-984a3fbc6c74 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:39.893163 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171b3a89-454a-473c-b7ad-984a3fbc6c74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:39.895304 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171b3a89-454a-473c-b7ad-984a3fbc6c74 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:39.927916 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1108/2237] 10.222.0.99 () {60 vars in 1152 bytes} [Thu Jul 14 15:38:39 2022] POST /identity/v3/projects => generated 383 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:38:39.932064 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a597d93a-f14b-4aca-b62c-fc1d62247223 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:39.946712 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a597d93a-f14b-4aca-b62c-fc1d62247223 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:40.013532 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c87e913-d0a1-4140-9fbb-4bd134749e5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:40.014220 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c87e913-d0a1-4140-9fbb-4bd134749e5c admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:40.014511 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c87e913-d0a1-4140-9fbb-4bd134749e5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:40.014794 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c87e913-d0a1-4140-9fbb-4bd134749e5c admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:40.015511 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c87e913-d0a1-4140-9fbb-4bd134749e5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:40.018152 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c87e913-d0a1-4140-9fbb-4bd134749e5c admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:40.034462 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171b3a89-454a-473c-b7ad-984a3fbc6c74 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:40.042039 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1130/2238] 10.222.0.99 () {60 vars in 1143 bytes} [Thu Jul 14 15:38:39 2022] POST /identity/v3/users => generated 292 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 15:38:40.045828 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c87e913-d0a1-4140-9fbb-4bd134749e5c admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:40.048873 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171b3a89-454a-473c-b7ad-984a3fbc6c74 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:40.053668 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2255beb0-f11f-47a1-a1b0-bbe5d18007df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:40.054551 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2255beb0-f11f-47a1-a1b0-bbe5d18007df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:40.054839 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2255beb0-f11f-47a1-a1b0-bbe5d18007df admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:40.055265 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2255beb0-f11f-47a1-a1b0-bbe5d18007df admin admin] PATH_INFO: `/v3/users/caa8148a3163453881214e833d56c043` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:40.056022 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c87e913-d0a1-4140-9fbb-4bd134749e5c admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:40.056557 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2255beb0-f11f-47a1-a1b0-bbe5d18007df admin admin] RBAC: Authorizing `identity:delete_user(user_id=caa8148a3163453881214e833d56c043)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:40.058495 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2255beb0-f11f-47a1-a1b0-bbe5d18007df admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:40.060167 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:40.060756 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af64f173-9b53-474d-9ef4-caecceaa9028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:40.061495 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af64f173-9b53-474d-9ef4-caecceaa9028 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:40.061790 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af64f173-9b53-474d-9ef4-caecceaa9028 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:40.062071 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af64f173-9b53-474d-9ef4-caecceaa9028 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:40.062839 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af64f173-9b53-474d-9ef4-caecceaa9028 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:40.064615 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af64f173-9b53-474d-9ef4-caecceaa9028 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:40.075176 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1131/2239] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:38:40 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 15:38:40.080598 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af64f173-9b53-474d-9ef4-caecceaa9028 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:40.086024 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:40.091754 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af64f173-9b53-474d-9ef4-caecceaa9028 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:40.102694 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da49baf-8283-4c67-9a44-2630adf51a35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:40.103388 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da49baf-8283-4c67-9a44-2630adf51a35 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:40.103709 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da49baf-8283-4c67-9a44-2630adf51a35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:40.104000 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da49baf-8283-4c67-9a44-2630adf51a35 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:40.104891 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da49baf-8283-4c67-9a44-2630adf51a35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:40.106829 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da49baf-8283-4c67-9a44-2630adf51a35 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:40.120579 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1132/2240] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:38:40 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:38:40.124750 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da49baf-8283-4c67-9a44-2630adf51a35 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:40.126094 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2255beb0-f11f-47a1-a1b0-bbe5d18007df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'caa8148a3163453881214e833d56c043'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:40.135642 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2255beb0-f11f-47a1-a1b0-bbe5d18007df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'caa8148a3163453881214e833d56c043'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:40.136306 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da49baf-8283-4c67-9a44-2630adf51a35 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:40.143476 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e9c13d-4321-4fcc-84e6-69c51860243a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:40.144697 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e9c13d-4321-4fcc-84e6-69c51860243a admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:40.145418 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e9c13d-4321-4fcc-84e6-69c51860243a admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:40.145711 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e9c13d-4321-4fcc-84e6-69c51860243a admin admin] PATH_INFO: `/v3/projects/194a16f2ffd446b3b3e9742e45fd1934/users/88f2982e29ff47d9ab0c3d2a886218c6/roles/b8c98fddc0fd4436b577fd1d3361bc38` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:40.150967 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2255beb0-f11f-47a1-a1b0-bbe5d18007df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'caa8148a3163453881214e833d56c043'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:40.152235 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2255beb0-f11f-47a1-a1b0-bbe5d18007df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'caa8148a3163453881214e833d56c043'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:40.152870 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2255beb0-f11f-47a1-a1b0-bbe5d18007df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'caa8148a3163453881214e833d56c043'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:40.163436 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e9c13d-4321-4fcc-84e6-69c51860243a admin admin] RBAC: Authorizing `identity:create_grant(project_id=194a16f2ffd446b3b3e9742e45fd1934, user_id=88f2982e29ff47d9ab0c3d2a886218c6, role_id=b8c98fddc0fd4436b577fd1d3361bc38)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:40.166146 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e9c13d-4321-4fcc-84e6-69c51860243a admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:40.167755 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1109/2241] 10.222.0.99 () {58 vars in 1223 bytes} [Thu Jul 14 15:38:40 2022] DELETE /identity/v3/users/caa8148a3163453881214e833d56c043 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 15:38:40.172009 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceec58ce-8725-4d97-8777-dd4bad2143e0 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:40.172299 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceec58ce-8725-4d97-8777-dd4bad2143e0 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:40.172582 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceec58ce-8725-4d97-8777-dd4bad2143e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:40.184507 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1133/2242] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 15:38:40 2022] PUT /identity/v3/projects/194a16f2ffd446b3b3e9742e45fd1934/users/88f2982e29ff47d9ab0c3d2a886218c6/roles/b8c98fddc0fd4436b577fd1d3361bc38 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 15:38:40.191967 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-ceec58ce-8725-4d97-8777-dd4bad2143e0 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:38:40.193188 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ceec58ce-8725-4d97-8777-dd4bad2143e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:40.196221 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9a6511-592c-4e9b-924f-845e1f209084 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:40.196697 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9a6511-592c-4e9b-924f-845e1f209084 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:40.196825 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9a6511-592c-4e9b-924f-845e1f209084 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:40.225680 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-ac9a6511-592c-4e9b-924f-845e1f209084 None None] MFA Rules not processed for user `88f2982e29ff47d9ab0c3d2a886218c6`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:38:40.228172 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-ac9a6511-592c-4e9b-924f-845e1f209084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:40.264545 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1110/2243] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:38:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 15:38:40.275353 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceec58ce-8725-4d97-8777-dd4bad2143e0 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:40.276771 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ceec58ce-8725-4d97-8777-dd4bad2143e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:40.318713 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1134/2244] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:38:40 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 15:38:40.323203 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9a6511-592c-4e9b-924f-845e1f209084 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:40.324625 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-ac9a6511-592c-4e9b-924f-845e1f209084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:40.339726 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9a6511-592c-4e9b-924f-845e1f209084 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:40.344819 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497ff82e-86d9-4d8d-80e8-a845bce3cd4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:40.346049 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497ff82e-86d9-4d8d-80e8-a845bce3cd4a admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:40.346337 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497ff82e-86d9-4d8d-80e8-a845bce3cd4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:40.346622 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497ff82e-86d9-4d8d-80e8-a845bce3cd4a admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:40.347337 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497ff82e-86d9-4d8d-80e8-a845bce3cd4a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:40.349311 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497ff82e-86d9-4d8d-80e8-a845bce3cd4a admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:40.352279 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceec58ce-8725-4d97-8777-dd4bad2143e0 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:40.356725 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492f3f66-47d6-4385-ae2f-50b6a4be65f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:40.357024 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492f3f66-47d6-4385-ae2f-50b6a4be65f0 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:40.357315 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492f3f66-47d6-4385-ae2f-50b6a4be65f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:40.357600 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492f3f66-47d6-4385-ae2f-50b6a4be65f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:40.369147 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492f3f66-47d6-4385-ae2f-50b6a4be65f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:40.370612 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492f3f66-47d6-4385-ae2f-50b6a4be65f0 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:40.381476 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1135/2245] 10.222.0.99 () {60 vars in 1152 bytes} [Thu Jul 14 15:38:40 2022] POST /identity/v3/projects => generated 385 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 15:38:40.382397 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1111/2246] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:38:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:38:40.385565 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497ff82e-86d9-4d8d-80e8-a845bce3cd4a admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:40.461134 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52b70a7-4408-4f68-8555-c706dd94b056 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:40.461528 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52b70a7-4408-4f68-8555-c706dd94b056 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:40.461888 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52b70a7-4408-4f68-8555-c706dd94b056 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:40.476009 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497ff82e-86d9-4d8d-80e8-a845bce3cd4a admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:40.481755 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef8887c-e89b-4158-914f-b38b52161642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:40.481935 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-c52b70a7-4408-4f68-8555-c706dd94b056 None None] MFA Rules not processed for user `71bc270c6253426baa4f79638d466b20`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:38:40.482504 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef8887c-e89b-4158-914f-b38b52161642 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:40.482773 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef8887c-e89b-4158-914f-b38b52161642 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:40.483171 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef8887c-e89b-4158-914f-b38b52161642 admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:40.483937 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c52b70a7-4408-4f68-8555-c706dd94b056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:40.484189 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef8887c-e89b-4158-914f-b38b52161642 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:40.487137 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef8887c-e89b-4158-914f-b38b52161642 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:40.505616 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:40.512557 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1136/2247] 10.222.0.99 () {60 vars in 1143 bytes} [Thu Jul 14 15:38:40 2022] POST /identity/v3/users => generated 299 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 15:38:40.513810 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef8887c-e89b-4158-914f-b38b52161642 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:40.527571 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef8887c-e89b-4158-914f-b38b52161642 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:40.533392 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640f7f4d-74d3-4a91-98c4-222e74df3b04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:40.534083 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640f7f4d-74d3-4a91-98c4-222e74df3b04 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:40.534377 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640f7f4d-74d3-4a91-98c4-222e74df3b04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:40.534661 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640f7f4d-74d3-4a91-98c4-222e74df3b04 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:40.535406 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640f7f4d-74d3-4a91-98c4-222e74df3b04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:40.537091 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640f7f4d-74d3-4a91-98c4-222e74df3b04 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:40.547673 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1137/2248] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:38:40 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 15:38:40.552295 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640f7f4d-74d3-4a91-98c4-222e74df3b04 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:40.562761 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640f7f4d-74d3-4a91-98c4-222e74df3b04 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:40.568354 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1112/2249] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:38:40 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 15:38:40.574620 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaee44e4-93f0-41a5-879d-7dfb2bd39e8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:40.575547 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaee44e4-93f0-41a5-879d-7dfb2bd39e8b admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:40.575869 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaee44e4-93f0-41a5-879d-7dfb2bd39e8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:40.576154 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaee44e4-93f0-41a5-879d-7dfb2bd39e8b admin admin] PATH_INFO: `/v3/projects/b95683a167804b6289ed527db3fc8142/users/b81bb209bd02495b8e66b2dc858fd3ab/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:40.589911 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:40.595198 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaee44e4-93f0-41a5-879d-7dfb2bd39e8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b95683a167804b6289ed527db3fc8142, user_id=b81bb209bd02495b8e66b2dc858fd3ab, role_id=d0f6ba1314f14d9f9c816bdb5d0c3aac)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:40.600759 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaee44e4-93f0-41a5-879d-7dfb2bd39e8b admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:40.625587 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1138/2250] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 15:38:40 2022] PUT /identity/v3/projects/b95683a167804b6289ed527db3fc8142/users/b81bb209bd02495b8e66b2dc858fd3ab/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:38:40.629763 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52b70a7-4408-4f68-8555-c706dd94b056 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:40.645923 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52b70a7-4408-4f68-8555-c706dd94b056 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:40.710917 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72414b3c-9076-42e6-8d9b-270a6d9c911c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:40.711625 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72414b3c-9076-42e6-8d9b-270a6d9c911c admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:40.711912 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72414b3c-9076-42e6-8d9b-270a6d9c911c admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:40.712200 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72414b3c-9076-42e6-8d9b-270a6d9c911c admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:40.712939 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72414b3c-9076-42e6-8d9b-270a6d9c911c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:40.714542 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72414b3c-9076-42e6-8d9b-270a6d9c911c admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:40.725995 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1113/2251] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:38:40 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 15:38:40.730062 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaee44e4-93f0-41a5-879d-7dfb2bd39e8b admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:40.741455 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaee44e4-93f0-41a5-879d-7dfb2bd39e8b admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:40.746235 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff67359f-c9a9-411c-a535-bda0350089a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:40.747145 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff67359f-c9a9-411c-a535-bda0350089a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:40.747431 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff67359f-c9a9-411c-a535-bda0350089a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:40.747752 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff67359f-c9a9-411c-a535-bda0350089a1 admin admin] PATH_INFO: `/v3/projects/b95683a167804b6289ed527db3fc8142/users/b81bb209bd02495b8e66b2dc858fd3ab/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:40.749855 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff67359f-c9a9-411c-a535-bda0350089a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b95683a167804b6289ed527db3fc8142, user_id=b81bb209bd02495b8e66b2dc858fd3ab, role_id=d0f6ba1314f14d9f9c816bdb5d0c3aac)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:40.752750 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff67359f-c9a9-411c-a535-bda0350089a1 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:40.766298 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1139/2252] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 15:38:40 2022] PUT /identity/v3/projects/b95683a167804b6289ed527db3fc8142/users/b81bb209bd02495b8e66b2dc858fd3ab/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 15:38:40.777714 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647b635e-f18c-4699-a45a-7a7107ac86e6 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:40.777991 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647b635e-f18c-4699-a45a-7a7107ac86e6 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:40.778305 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647b635e-f18c-4699-a45a-7a7107ac86e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:40.810383 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-647b635e-f18c-4699-a45a-7a7107ac86e6 None None] MFA Rules not processed for user `b81bb209bd02495b8e66b2dc858fd3ab`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:38:40.811664 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-647b635e-f18c-4699-a45a-7a7107ac86e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:40.887528 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1114/2253] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:38:40 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:38:40.900124 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff67359f-c9a9-411c-a535-bda0350089a1 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:40.902137 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-ff67359f-c9a9-411c-a535-bda0350089a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:40.968634 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff67359f-c9a9-411c-a535-bda0350089a1 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:40.973430 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618f1baf-06b3-47aa-a7f9-61b50d4b506e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3e1a2b83f3d4d88ba5084dfaffd22f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:40.974687 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618f1baf-06b3-47aa-a7f9-61b50d4b506e service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:40.974975 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618f1baf-06b3-47aa-a7f9-61b50d4b506e service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:40.975257 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618f1baf-06b3-47aa-a7f9-61b50d4b506e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:40.977186 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-618f1baf-06b3-47aa-a7f9-61b50d4b506e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:41.051452 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647b635e-f18c-4699-a45a-7a7107ac86e6 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:41.054121 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-647b635e-f18c-4699-a45a-7a7107ac86e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:41.054317 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618f1baf-06b3-47aa-a7f9-61b50d4b506e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:41.055778 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618f1baf-06b3-47aa-a7f9-61b50d4b506e service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:41.072280 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1140/2254] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:38:40 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:38:41.151363 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:41.170676 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647b635e-f18c-4699-a45a-7a7107ac86e6 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:41.177215 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772d6a17-aea5-4827-a3ee-6cb134847b59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:41.178362 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772d6a17-aea5-4827-a3ee-6cb134847b59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:41.178362 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772d6a17-aea5-4827-a3ee-6cb134847b59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:41.178362 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772d6a17-aea5-4827-a3ee-6cb134847b59 admin admin] PATH_INFO: `/v3/projects/a69112f898d74cdea01cf1bae7dd30cd` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:41.179694 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772d6a17-aea5-4827-a3ee-6cb134847b59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a69112f898d74cdea01cf1bae7dd30cd)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:41.182328 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772d6a17-aea5-4827-a3ee-6cb134847b59 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:41.237735 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-772d6a17-aea5-4827-a3ee-6cb134847b59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a69112f898d74cdea01cf1bae7dd30cd', 'request_id': 'req-772d6a17-aea5-4827-a3ee-6cb134847b59'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:41.244829 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-772d6a17-aea5-4827-a3ee-6cb134847b59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a69112f898d74cdea01cf1bae7dd30cd', 'request_id': 'req-772d6a17-aea5-4827-a3ee-6cb134847b59'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:41.251221 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-772d6a17-aea5-4827-a3ee-6cb134847b59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a69112f898d74cdea01cf1bae7dd30cd', 'request_id': 'req-772d6a17-aea5-4827-a3ee-6cb134847b59'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:41.266834 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-772d6a17-aea5-4827-a3ee-6cb134847b59 admin admin] The token cache is being invalidate because project a69112f898d74cdea01cf1bae7dd30cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 15:38:41.267859 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-772d6a17-aea5-4827-a3ee-6cb134847b59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:41.268810 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-772d6a17-aea5-4827-a3ee-6cb134847b59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:41.270569 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1115/2255] 10.222.0.99 () {58 vars in 1232 bytes} [Thu Jul 14 15:38:41 2022] DELETE /identity/v3/projects/a69112f898d74cdea01cf1bae7dd30cd => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 15:38:41.274776 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618f1baf-06b3-47aa-a7f9-61b50d4b506e service cinder] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:41.292281 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618f1baf-06b3-47aa-a7f9-61b50d4b506e service cinder] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:41.363479 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db005bb-5a0b-4cf1-967c-7a7449351255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:41.364896 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db005bb-5a0b-4cf1-967c-7a7449351255 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:41.365554 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db005bb-5a0b-4cf1-967c-7a7449351255 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:41.365930 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db005bb-5a0b-4cf1-967c-7a7449351255 admin admin] PATH_INFO: `/v3/users/0eefe94420b445b6adc9284165ea34c3` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:41.367881 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db005bb-5a0b-4cf1-967c-7a7449351255 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0eefe94420b445b6adc9284165ea34c3)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:41.370322 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db005bb-5a0b-4cf1-967c-7a7449351255 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:41.419305 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:41.446512 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-8db005bb-5a0b-4cf1-967c-7a7449351255 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0eefe94420b445b6adc9284165ea34c3'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:41.461540 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-8db005bb-5a0b-4cf1-967c-7a7449351255 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0eefe94420b445b6adc9284165ea34c3'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:41.477945 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-8db005bb-5a0b-4cf1-967c-7a7449351255 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0eefe94420b445b6adc9284165ea34c3'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:41.478678 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-8db005bb-5a0b-4cf1-967c-7a7449351255 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0eefe94420b445b6adc9284165ea34c3'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:41.479306 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-8db005bb-5a0b-4cf1-967c-7a7449351255 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0eefe94420b445b6adc9284165ea34c3'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:41.497483 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1141/2256] 10.222.0.99 () {58 vars in 1223 bytes} [Thu Jul 14 15:38:41 2022] DELETE /identity/v3/users/0eefe94420b445b6adc9284165ea34c3 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:38:41.901130 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772d6a17-aea5-4827-a3ee-6cb134847b59 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:41.903022 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-772d6a17-aea5-4827-a3ee-6cb134847b59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:41.917684 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772d6a17-aea5-4827-a3ee-6cb134847b59 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:41.987214 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af8a1d2-01a0-4c53-8fba-7fdf90fb6cf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:41.988623 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af8a1d2-01a0-4c53-8fba-7fdf90fb6cf4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:41.989042 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af8a1d2-01a0-4c53-8fba-7fdf90fb6cf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:41.989315 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af8a1d2-01a0-4c53-8fba-7fdf90fb6cf4 admin admin] PATH_INFO: `/v3/projects/d0f63223785242fba7905a51cd2c1d0c` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:41.990787 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af8a1d2-01a0-4c53-8fba-7fdf90fb6cf4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0f63223785242fba7905a51cd2c1d0c)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:41.992880 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af8a1d2-01a0-4c53-8fba-7fdf90fb6cf4 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:42.045517 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-0af8a1d2-01a0-4c53-8fba-7fdf90fb6cf4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0f63223785242fba7905a51cd2c1d0c', 'request_id': 'req-0af8a1d2-01a0-4c53-8fba-7fdf90fb6cf4'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:42.052372 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-0af8a1d2-01a0-4c53-8fba-7fdf90fb6cf4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0f63223785242fba7905a51cd2c1d0c', 'request_id': 'req-0af8a1d2-01a0-4c53-8fba-7fdf90fb6cf4'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:42.057983 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-0af8a1d2-01a0-4c53-8fba-7fdf90fb6cf4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0f63223785242fba7905a51cd2c1d0c', 'request_id': 'req-0af8a1d2-01a0-4c53-8fba-7fdf90fb6cf4'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:42.071198 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-0af8a1d2-01a0-4c53-8fba-7fdf90fb6cf4 admin admin] The token cache is being invalidate because project d0f63223785242fba7905a51cd2c1d0c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 15:38:42.071489 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-0af8a1d2-01a0-4c53-8fba-7fdf90fb6cf4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:42.072132 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-0af8a1d2-01a0-4c53-8fba-7fdf90fb6cf4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:42.073703 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1116/2257] 10.222.0.99 () {58 vars in 1232 bytes} [Thu Jul 14 15:38:41 2022] DELETE /identity/v3/projects/d0f63223785242fba7905a51cd2c1d0c => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:38:42.079417 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485578db-ae93-419e-a056-d546c0918bb8 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:42.079803 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485578db-ae93-419e-a056-d546c0918bb8 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:42.080060 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485578db-ae93-419e-a056-d546c0918bb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:42.099582 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-485578db-ae93-419e-a056-d546c0918bb8 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:38:42.100826 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-485578db-ae93-419e-a056-d546c0918bb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:42.167968 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1142/2258] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:38:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:38:42.177291 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a97285f-e898-4bf6-b23f-9a509f3db357 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:42.177500 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a97285f-e898-4bf6-b23f-9a509f3db357 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:42.177782 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a97285f-e898-4bf6-b23f-9a509f3db357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:42.197868 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-4a97285f-e898-4bf6-b23f-9a509f3db357 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:38:42.199112 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-4a97285f-e898-4bf6-b23f-9a509f3db357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:42.207353 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1117/2259] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:38:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:38:42.211821 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485578db-ae93-419e-a056-d546c0918bb8 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:42.221908 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485578db-ae93-419e-a056-d546c0918bb8 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:42.226569 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357a1757-32af-424c-ab73-f7e8ada99aa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:42.227236 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357a1757-32af-424c-ab73-f7e8ada99aa8 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:42.227526 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357a1757-32af-424c-ab73-f7e8ada99aa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:42.227844 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357a1757-32af-424c-ab73-f7e8ada99aa8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:42.228596 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357a1757-32af-424c-ab73-f7e8ada99aa8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:38:42.228901 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357a1757-32af-424c-ab73-f7e8ada99aa8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:42.230541 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357a1757-32af-424c-ab73-f7e8ada99aa8 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:42.242824 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1143/2260] 10.222.0.99 () {58 vars in 1152 bytes} [Thu Jul 14 15:38:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:38:42.247259 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a97285f-e898-4bf6-b23f-9a509f3db357 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:42.252362 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:42.258957 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a97285f-e898-4bf6-b23f-9a509f3db357 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:42.264955 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c0b61f-4e04-46f9-8e10-095c648b356d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:42.265623 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c0b61f-4e04-46f9-8e10-095c648b356d admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:42.265914 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c0b61f-4e04-46f9-8e10-095c648b356d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:42.266196 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c0b61f-4e04-46f9-8e10-095c648b356d admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:42.266981 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c0b61f-4e04-46f9-8e10-095c648b356d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:42.269195 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c0b61f-4e04-46f9-8e10-095c648b356d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:42.295988 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1118/2261] 10.222.0.99 () {60 vars in 1152 bytes} [Thu Jul 14 15:38:42 2022] POST /identity/v3/projects => generated 361 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:38:42.301160 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357a1757-32af-424c-ab73-f7e8ada99aa8 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:42.311186 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357a1757-32af-424c-ab73-f7e8ada99aa8 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:42.371182 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5b05db-b98d-4dfe-b278-206b1f899cb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:42.371890 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5b05db-b98d-4dfe-b278-206b1f899cb8 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:42.372181 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5b05db-b98d-4dfe-b278-206b1f899cb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:42.372465 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5b05db-b98d-4dfe-b278-206b1f899cb8 admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:42.373175 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5b05db-b98d-4dfe-b278-206b1f899cb8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:42.375094 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5b05db-b98d-4dfe-b278-206b1f899cb8 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:42.395619 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1144/2262] 10.222.0.99 () {60 vars in 1143 bytes} [Thu Jul 14 15:38:42 2022] POST /identity/v3/users => generated 281 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:38:42.400005 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c0b61f-4e04-46f9-8e10-095c648b356d admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:42.411029 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c0b61f-4e04-46f9-8e10-095c648b356d admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:42.415745 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c475088c-4236-42a2-93c9-3bd859acd8fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:42.416410 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c475088c-4236-42a2-93c9-3bd859acd8fd admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:42.416699 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c475088c-4236-42a2-93c9-3bd859acd8fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:42.416981 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c475088c-4236-42a2-93c9-3bd859acd8fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:42.417720 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c475088c-4236-42a2-93c9-3bd859acd8fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:42.419313 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c475088c-4236-42a2-93c9-3bd859acd8fd admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:42.430839 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1119/2263] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:38:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:38:42.435990 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5b05db-b98d-4dfe-b278-206b1f899cb8 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:42.446883 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5b05db-b98d-4dfe-b278-206b1f899cb8 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:42.451532 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b32786b-c29a-4130-99c8-a7871a73bd5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:42.452233 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b32786b-c29a-4130-99c8-a7871a73bd5d admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:42.452521 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b32786b-c29a-4130-99c8-a7871a73bd5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:42.452848 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b32786b-c29a-4130-99c8-a7871a73bd5d admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:42.453585 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b32786b-c29a-4130-99c8-a7871a73bd5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:42.456104 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b32786b-c29a-4130-99c8-a7871a73bd5d admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:42.466249 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1145/2264] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:38:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:38:42.470792 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c475088c-4236-42a2-93c9-3bd859acd8fd admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:42.481160 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c475088c-4236-42a2-93c9-3bd859acd8fd admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:42.486667 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2e5999-4524-4af8-89e7-3ee45bbee556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:42.487563 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2e5999-4524-4af8-89e7-3ee45bbee556 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:42.487940 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2e5999-4524-4af8-89e7-3ee45bbee556 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:42.488216 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2e5999-4524-4af8-89e7-3ee45bbee556 admin admin] PATH_INFO: `/v3/projects/455c85fb9b5e45218ae7076a682ecaba/users/26f55cdaab19406b82d0ce4b4886980d/roles/b8c98fddc0fd4436b577fd1d3361bc38` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:42.506259 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2e5999-4524-4af8-89e7-3ee45bbee556 admin admin] RBAC: Authorizing `identity:create_grant(project_id=455c85fb9b5e45218ae7076a682ecaba, user_id=26f55cdaab19406b82d0ce4b4886980d, role_id=b8c98fddc0fd4436b577fd1d3361bc38)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:42.509246 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2e5999-4524-4af8-89e7-3ee45bbee556 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:42.525757 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1120/2265] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 15:38:42 2022] PUT /identity/v3/projects/455c85fb9b5e45218ae7076a682ecaba/users/26f55cdaab19406b82d0ce4b4886980d/roles/b8c98fddc0fd4436b577fd1d3361bc38 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:38:42.537327 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed4c431-8181-438c-8fc7-4d7d7c630465 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:42.537627 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed4c431-8181-438c-8fc7-4d7d7c630465 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:42.537948 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed4c431-8181-438c-8fc7-4d7d7c630465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:42.567575 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-6ed4c431-8181-438c-8fc7-4d7d7c630465 None None] MFA Rules not processed for user `26f55cdaab19406b82d0ce4b4886980d`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:38:42.570465 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-6ed4c431-8181-438c-8fc7-4d7d7c630465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:42.575374 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:42.650684 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2e5999-4524-4af8-89e7-3ee45bbee556 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:42.652188 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-1d2e5999-4524-4af8-89e7-3ee45bbee556 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:42.661505 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1146/2266] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:38:42 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:38:42.672409 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed4c431-8181-438c-8fc7-4d7d7c630465 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:42.674425 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-6ed4c431-8181-438c-8fc7-4d7d7c630465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:42.718938 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2e5999-4524-4af8-89e7-3ee45bbee556 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:42.723423 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d3facf-d510-4355-b7b9-09f104cd1685 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3e1a2b83f3d4d88ba5084dfaffd22f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:42.724203 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d3facf-d510-4355-b7b9-09f104cd1685 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:42.724555 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d3facf-d510-4355-b7b9-09f104cd1685 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:42.724918 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d3facf-d510-4355-b7b9-09f104cd1685 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:42.726918 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-87d3facf-d510-4355-b7b9-09f104cd1685 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:42.748928 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed4c431-8181-438c-8fc7-4d7d7c630465 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:42.756881 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8425ed66-94b3-48ec-821b-51dd2fa69764 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3e1a2b83f3d4d88ba5084dfaffd22f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:42.757795 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8425ed66-94b3-48ec-821b-51dd2fa69764 service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:42.758083 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8425ed66-94b3-48ec-821b-51dd2fa69764 service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:42.758364 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8425ed66-94b3-48ec-821b-51dd2fa69764 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:42.769383 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8425ed66-94b3-48ec-821b-51dd2fa69764 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:42.771065 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8425ed66-94b3-48ec-821b-51dd2fa69764 service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:42.800446 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1147/2267] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:38:42 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:38:42.845135 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d3facf-d510-4355-b7b9-09f104cd1685 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:42.846984 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d3facf-d510-4355-b7b9-09f104cd1685 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:42.880374 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1121/2268] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:38:42 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 15:38:43.664259 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8425ed66-94b3-48ec-821b-51dd2fa69764 service cinder] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:43.674730 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8425ed66-94b3-48ec-821b-51dd2fa69764 service cinder] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:43.680672 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f178c03-d8e1-45fa-980f-663a0451d403 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3e1a2b83f3d4d88ba5084dfaffd22f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:43.681669 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f178c03-d8e1-45fa-980f-663a0451d403 service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:43.682037 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f178c03-d8e1-45fa-980f-663a0451d403 service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:43.682378 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f178c03-d8e1-45fa-980f-663a0451d403 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:43.685030 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-0f178c03-d8e1-45fa-980f-663a0451d403 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:43.768599 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f178c03-d8e1-45fa-980f-663a0451d403 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:43.769976 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f178c03-d8e1-45fa-980f-663a0451d403 service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:43.787720 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1148/2269] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:38:43 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:38:47.780222 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d3facf-d510-4355-b7b9-09f104cd1685 service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:47.781717 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-87d3facf-d510-4355-b7b9-09f104cd1685 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:47.783699 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f178c03-d8e1-45fa-980f-663a0451d403 service cinder] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:47.785965 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-0f178c03-d8e1-45fa-980f-663a0451d403 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:47.844539 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:47.855107 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d3facf-d510-4355-b7b9-09f104cd1685 service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:47.859882 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9e4c65-0ac9-4346-90ec-45e73f6096a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:47.860930 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9e4c65-0ac9-4346-90ec-45e73f6096a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:47.860930 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9e4c65-0ac9-4346-90ec-45e73f6096a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:47.861540 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9e4c65-0ac9-4346-90ec-45e73f6096a4 admin admin] PATH_INFO: `/v3/users/485f3a4cdf384a7e9de6e578f6dee765` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:47.862643 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9e4c65-0ac9-4346-90ec-45e73f6096a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=485f3a4cdf384a7e9de6e578f6dee765)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:47.864659 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9e4c65-0ac9-4346-90ec-45e73f6096a4 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:47.865082 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f178c03-d8e1-45fa-980f-663a0451d403 service cinder] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:47.866321 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:47.870234 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4d0759-280f-4e82-8bb5-d1cd56fb6226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:47.871318 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4d0759-280f-4e82-8bb5-d1cd56fb6226 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:47.872553 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4d0759-280f-4e82-8bb5-d1cd56fb6226 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:47.872839 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4d0759-280f-4e82-8bb5-d1cd56fb6226 admin admin] PATH_INFO: `/v3/users/f69abc1537be4ca99a2741ddbaca6faa` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:47.874214 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4d0759-280f-4e82-8bb5-d1cd56fb6226 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f69abc1537be4ca99a2741ddbaca6faa)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:47.876733 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4d0759-280f-4e82-8bb5-d1cd56fb6226 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:47.917260 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-ff9e4c65-0ac9-4346-90ec-45e73f6096a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '485f3a4cdf384a7e9de6e578f6dee765'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:47.924558 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-ff9e4c65-0ac9-4346-90ec-45e73f6096a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '485f3a4cdf384a7e9de6e578f6dee765'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:47.927684 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-8c4d0759-280f-4e82-8bb5-d1cd56fb6226 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f69abc1537be4ca99a2741ddbaca6faa'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:47.935538 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-8c4d0759-280f-4e82-8bb5-d1cd56fb6226 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f69abc1537be4ca99a2741ddbaca6faa'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:47.937222 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-ff9e4c65-0ac9-4346-90ec-45e73f6096a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '485f3a4cdf384a7e9de6e578f6dee765'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:47.937887 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-ff9e4c65-0ac9-4346-90ec-45e73f6096a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '485f3a4cdf384a7e9de6e578f6dee765'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:47.939036 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-ff9e4c65-0ac9-4346-90ec-45e73f6096a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '485f3a4cdf384a7e9de6e578f6dee765'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:47.948602 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-8c4d0759-280f-4e82-8bb5-d1cd56fb6226 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f69abc1537be4ca99a2741ddbaca6faa'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:47.949253 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-8c4d0759-280f-4e82-8bb5-d1cd56fb6226 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f69abc1537be4ca99a2741ddbaca6faa'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:47.949871 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-8c4d0759-280f-4e82-8bb5-d1cd56fb6226 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f69abc1537be4ca99a2741ddbaca6faa'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:47.952358 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1122/2270] 10.222.0.99 () {58 vars in 1223 bytes} [Thu Jul 14 15:38:47 2022] DELETE /identity/v3/users/485f3a4cdf384a7e9de6e578f6dee765 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:38:47.957035 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d4746c-d0a0-4666-b7da-7a2211eb6904 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:47.957330 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d4746c-d0a0-4666-b7da-7a2211eb6904 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:47.957612 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d4746c-d0a0-4666-b7da-7a2211eb6904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:47.965062 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1149/2271] 10.222.0.99 () {58 vars in 1223 bytes} [Thu Jul 14 15:38:47 2022] DELETE /identity/v3/users/f69abc1537be4ca99a2741ddbaca6faa => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 15:38:47.969330 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa3600d-d62e-432a-b1f0-ac929ef1aa5c None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:47.969630 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa3600d-d62e-432a-b1f0-ac929ef1aa5c None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:47.970015 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa3600d-d62e-432a-b1f0-ac929ef1aa5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:47.975136 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-95d4746c-d0a0-4666-b7da-7a2211eb6904 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:38:47.976376 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-95d4746c-d0a0-4666-b7da-7a2211eb6904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:47.987870 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-cfa3600d-d62e-432a-b1f0-ac929ef1aa5c None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:38:47.989102 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-cfa3600d-d62e-432a-b1f0-ac929ef1aa5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:48.043227 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1123/2272] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:38:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 15:38:48.050007 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1150/2273] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:38:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 15:38:48.053242 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d4746c-d0a0-4666-b7da-7a2211eb6904 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:48.054729 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-95d4746c-d0a0-4666-b7da-7a2211eb6904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:48.058710 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa3600d-d62e-432a-b1f0-ac929ef1aa5c None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:48.060171 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-cfa3600d-d62e-432a-b1f0-ac929ef1aa5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:48.115020 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d4746c-d0a0-4666-b7da-7a2211eb6904 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:48.118876 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bb15c1-3ee4-498d-a452-91a555af7a03 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:48.119662 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bb15c1-3ee4-498d-a452-91a555af7a03 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:48.120023 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bb15c1-3ee4-498d-a452-91a555af7a03 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:48.120269 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa3600d-d62e-432a-b1f0-ac929ef1aa5c None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:48.120396 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bb15c1-3ee4-498d-a452-91a555af7a03 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:48.123921 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384ea0aa-188d-4db2-b69c-9f3d485d59c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:48.124572 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384ea0aa-188d-4db2-b69c-9f3d485d59c2 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:48.124869 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384ea0aa-188d-4db2-b69c-9f3d485d59c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:48.125180 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384ea0aa-188d-4db2-b69c-9f3d485d59c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:48.131072 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bb15c1-3ee4-498d-a452-91a555af7a03 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:48.132572 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bb15c1-3ee4-498d-a452-91a555af7a03 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:48.135235 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384ea0aa-188d-4db2-b69c-9f3d485d59c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:48.136739 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384ea0aa-188d-4db2-b69c-9f3d485d59c2 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:48.143530 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1124/2274] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:38:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 15:38:48.148533 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1151/2275] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:38:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 15:38:48.841315 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bb15c1-3ee4-498d-a452-91a555af7a03 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:48.842751 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a9bb15c1-3ee4-498d-a452-91a555af7a03 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:48.862411 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bb15c1-3ee4-498d-a452-91a555af7a03 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:48.868776 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d2fa66-7863-4b47-8b74-428fdf41ecb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:48.869895 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d2fa66-7863-4b47-8b74-428fdf41ecb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:48.870214 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d2fa66-7863-4b47-8b74-428fdf41ecb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:48.870708 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d2fa66-7863-4b47-8b74-428fdf41ecb1 admin admin] PATH_INFO: `/v3/projects/1d871080e80348f48bc10813e9649548` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:48.872231 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d2fa66-7863-4b47-8b74-428fdf41ecb1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d871080e80348f48bc10813e9649548)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:48.874691 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d2fa66-7863-4b47-8b74-428fdf41ecb1 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:48.885516 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384ea0aa-188d-4db2-b69c-9f3d485d59c2 service neutron] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:48.886876 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-384ea0aa-188d-4db2-b69c-9f3d485d59c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:48.898048 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:48.900356 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:48.905284 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384ea0aa-188d-4db2-b69c-9f3d485d59c2 service neutron] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:48.924748 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803a0bb2-9d30-4c36-8be3-63466a9ba2a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:48.925897 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803a0bb2-9d30-4c36-8be3-63466a9ba2a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:48.926354 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803a0bb2-9d30-4c36-8be3-63466a9ba2a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:48.926632 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803a0bb2-9d30-4c36-8be3-63466a9ba2a5 admin admin] PATH_INFO: `/v3/projects/a6fb7d0070ab49f1a4e8224179d09e4b` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:48.927974 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803a0bb2-9d30-4c36-8be3-63466a9ba2a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6fb7d0070ab49f1a4e8224179d09e4b)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:48.929904 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803a0bb2-9d30-4c36-8be3-63466a9ba2a5 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:48.947048 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-32d2fa66-7863-4b47-8b74-428fdf41ecb1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d871080e80348f48bc10813e9649548', 'request_id': 'req-32d2fa66-7863-4b47-8b74-428fdf41ecb1'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:48.954696 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-32d2fa66-7863-4b47-8b74-428fdf41ecb1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d871080e80348f48bc10813e9649548', 'request_id': 'req-32d2fa66-7863-4b47-8b74-428fdf41ecb1'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:48.962442 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-32d2fa66-7863-4b47-8b74-428fdf41ecb1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d871080e80348f48bc10813e9649548', 'request_id': 'req-32d2fa66-7863-4b47-8b74-428fdf41ecb1'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:48.976727 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-32d2fa66-7863-4b47-8b74-428fdf41ecb1 admin admin] The token cache is being invalidate because project 1d871080e80348f48bc10813e9649548 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 15:38:48.976937 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-32d2fa66-7863-4b47-8b74-428fdf41ecb1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:48.977792 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-32d2fa66-7863-4b47-8b74-428fdf41ecb1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:48.979626 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1125/2276] 10.222.0.99 () {58 vars in 1232 bytes} [Thu Jul 14 15:38:48 2022] DELETE /identity/v3/projects/1d871080e80348f48bc10813e9649548 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:38:48.981439 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-803a0bb2-9d30-4c36-8be3-63466a9ba2a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6fb7d0070ab49f1a4e8224179d09e4b', 'request_id': 'req-803a0bb2-9d30-4c36-8be3-63466a9ba2a5'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:48.983134 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d2fa66-7863-4b47-8b74-428fdf41ecb1 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:48.984473 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-32d2fa66-7863-4b47-8b74-428fdf41ecb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:48.989282 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-803a0bb2-9d30-4c36-8be3-63466a9ba2a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6fb7d0070ab49f1a4e8224179d09e4b', 'request_id': 'req-803a0bb2-9d30-4c36-8be3-63466a9ba2a5'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:48.999624 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-803a0bb2-9d30-4c36-8be3-63466a9ba2a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6fb7d0070ab49f1a4e8224179d09e4b', 'request_id': 'req-803a0bb2-9d30-4c36-8be3-63466a9ba2a5'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:49.013716 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-803a0bb2-9d30-4c36-8be3-63466a9ba2a5 admin admin] The token cache is being invalidate because project a6fb7d0070ab49f1a4e8224179d09e4b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 15:38:49.014012 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-803a0bb2-9d30-4c36-8be3-63466a9ba2a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:49.014725 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-803a0bb2-9d30-4c36-8be3-63466a9ba2a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:49.016353 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1152/2277] 10.222.0.99 () {58 vars in 1232 bytes} [Thu Jul 14 15:38:48 2022] DELETE /identity/v3/projects/a6fb7d0070ab49f1a4e8224179d09e4b => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:38:49.021292 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9cdb62-a98f-4609-b4e5-1206cffc8dfe None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:49.021590 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9cdb62-a98f-4609-b4e5-1206cffc8dfe None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:49.022129 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9cdb62-a98f-4609-b4e5-1206cffc8dfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:49.041167 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-0d9cdb62-a98f-4609-b4e5-1206cffc8dfe None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:38:49.042381 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-0d9cdb62-a98f-4609-b4e5-1206cffc8dfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:49.054911 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d2fa66-7863-4b47-8b74-428fdf41ecb1 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:49.102975 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1153/2278] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:38:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 15:38:49.111045 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eac006-0186-41d9-a4cf-4249e36cebad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:49.111261 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13b5969-88d7-4283-8428-893b197780ae None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:49.111470 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13b5969-88d7-4283-8428-893b197780ae None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:49.111794 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13b5969-88d7-4283-8428-893b197780ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:49.112120 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eac006-0186-41d9-a4cf-4249e36cebad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:49.112404 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eac006-0186-41d9-a4cf-4249e36cebad admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:49.112681 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eac006-0186-41d9-a4cf-4249e36cebad admin admin] PATH_INFO: `/v3/users/88f2982e29ff47d9ab0c3d2a886218c6` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:49.113878 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98eac006-0186-41d9-a4cf-4249e36cebad admin admin] RBAC: Authorizing `identity:delete_user(user_id=88f2982e29ff47d9ab0c3d2a886218c6)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:49.115834 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98eac006-0186-41d9-a4cf-4249e36cebad admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:49.130044 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-d13b5969-88d7-4283-8428-893b197780ae None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:38:49.131296 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-d13b5969-88d7-4283-8428-893b197780ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:49.144997 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1154/2279] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:38:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 15:38:49.146051 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13b5969-88d7-4283-8428-893b197780ae None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:49.158485 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13b5969-88d7-4283-8428-893b197780ae None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:49.163360 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111c020e-2ebe-4f83-8aee-7c7444f8cf2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:49.164069 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111c020e-2ebe-4f83-8aee-7c7444f8cf2a admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:49.164406 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111c020e-2ebe-4f83-8aee-7c7444f8cf2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:49.164651 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111c020e-2ebe-4f83-8aee-7c7444f8cf2a admin admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:49.165417 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111c020e-2ebe-4f83-8aee-7c7444f8cf2a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:38:49.165736 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111c020e-2ebe-4f83-8aee-7c7444f8cf2a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:49.167522 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111c020e-2ebe-4f83-8aee-7c7444f8cf2a admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:49.168495 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-98eac006-0186-41d9-a4cf-4249e36cebad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88f2982e29ff47d9ab0c3d2a886218c6'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:49.178237 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-98eac006-0186-41d9-a4cf-4249e36cebad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88f2982e29ff47d9ab0c3d2a886218c6'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:49.179752 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:49.190021 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1155/2280] 10.222.0.99 () {58 vars in 1152 bytes} [Thu Jul 14 15:38:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:38:49.191582 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111c020e-2ebe-4f83-8aee-7c7444f8cf2a admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:49.198885 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-98eac006-0186-41d9-a4cf-4249e36cebad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88f2982e29ff47d9ab0c3d2a886218c6'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:49.199689 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-98eac006-0186-41d9-a4cf-4249e36cebad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88f2982e29ff47d9ab0c3d2a886218c6'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:49.200320 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-98eac006-0186-41d9-a4cf-4249e36cebad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88f2982e29ff47d9ab0c3d2a886218c6'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:49.203028 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111c020e-2ebe-4f83-8aee-7c7444f8cf2a admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:49.208424 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94393a7-f647-4843-8e6c-2ac6c1addef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:49.209259 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94393a7-f647-4843-8e6c-2ac6c1addef1 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:49.209551 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94393a7-f647-4843-8e6c-2ac6c1addef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:49.209836 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94393a7-f647-4843-8e6c-2ac6c1addef1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:49.210541 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94393a7-f647-4843-8e6c-2ac6c1addef1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:49.213054 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94393a7-f647-4843-8e6c-2ac6c1addef1 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:49.217523 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1126/2281] 10.222.0.99 () {58 vars in 1223 bytes} [Thu Jul 14 15:38:48 2022] DELETE /identity/v3/users/88f2982e29ff47d9ab0c3d2a886218c6 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 15:38:49.247085 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1156/2282] 10.222.0.99 () {60 vars in 1152 bytes} [Thu Jul 14 15:38:49 2022] POST /identity/v3/projects => generated 375 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 15:38:49.251772 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eac006-0186-41d9-a4cf-4249e36cebad admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:49.253174 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-98eac006-0186-41d9-a4cf-4249e36cebad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:49.334451 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eac006-0186-41d9-a4cf-4249e36cebad admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:49.401248 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13724f58-609c-45ea-acf9-ab9204cc141c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:49.401938 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13724f58-609c-45ea-acf9-ab9204cc141c admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:49.402226 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13724f58-609c-45ea-acf9-ab9204cc141c admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:49.402511 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13724f58-609c-45ea-acf9-ab9204cc141c admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:49.403386 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13724f58-609c-45ea-acf9-ab9204cc141c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:49.405501 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13724f58-609c-45ea-acf9-ab9204cc141c admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:49.427688 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1127/2283] 10.222.0.99 () {60 vars in 1143 bytes} [Thu Jul 14 15:38:49 2022] POST /identity/v3/users => generated 288 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:38:49.431510 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94393a7-f647-4843-8e6c-2ac6c1addef1 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:49.442465 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94393a7-f647-4843-8e6c-2ac6c1addef1 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:49.447135 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65897fcd-31f1-454b-b4c2-d1eee76c9009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:49.447835 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65897fcd-31f1-454b-b4c2-d1eee76c9009 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:49.448427 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65897fcd-31f1-454b-b4c2-d1eee76c9009 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:49.448427 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65897fcd-31f1-454b-b4c2-d1eee76c9009 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:49.449118 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65897fcd-31f1-454b-b4c2-d1eee76c9009 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:49.450724 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65897fcd-31f1-454b-b4c2-d1eee76c9009 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:49.464264 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1157/2284] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:38:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 15:38:49.465462 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13724f58-609c-45ea-acf9-ab9204cc141c admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:49.476243 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13724f58-609c-45ea-acf9-ab9204cc141c admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:49.486608 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef15ccc5-6923-4613-a4dc-4731f9e209b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:49.487394 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef15ccc5-6923-4613-a4dc-4731f9e209b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:49.487782 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef15ccc5-6923-4613-a4dc-4731f9e209b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:49.488130 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef15ccc5-6923-4613-a4dc-4731f9e209b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:49.488812 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef15ccc5-6923-4613-a4dc-4731f9e209b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:49.490424 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef15ccc5-6923-4613-a4dc-4731f9e209b6 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:49.505285 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1128/2285] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:38:49 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 15:38:49.508411 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65897fcd-31f1-454b-b4c2-d1eee76c9009 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:49.519550 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65897fcd-31f1-454b-b4c2-d1eee76c9009 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:49.522024 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef15ccc5-6923-4613-a4dc-4731f9e209b6 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:49.525037 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ef15ccc5-6923-4613-a4dc-4731f9e209b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:49.525797 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c0e52d-db7b-44bd-b889-a1cd67e8e6c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:49.526482 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c0e52d-db7b-44bd-b889-a1cd67e8e6c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:49.526775 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c0e52d-db7b-44bd-b889-a1cd67e8e6c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:49.527057 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c0e52d-db7b-44bd-b889-a1cd67e8e6c8 admin admin] PATH_INFO: `/v3/projects/a8c9fce785074a5fbf2b6b0efe538c59/users/f0c0d784aea54218bfec1485c1d79c6b/roles/b8c98fddc0fd4436b577fd1d3361bc38` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:49.546837 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef15ccc5-6923-4613-a4dc-4731f9e209b6 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:49.552408 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731d20ed-627a-4a7b-93a3-9cc3dce48661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:49.553242 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731d20ed-627a-4a7b-93a3-9cc3dce48661 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:49.553764 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731d20ed-627a-4a7b-93a3-9cc3dce48661 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:49.553764 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731d20ed-627a-4a7b-93a3-9cc3dce48661 admin admin] PATH_INFO: `/v3/projects/194a16f2ffd446b3b3e9742e45fd1934` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:49.554981 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731d20ed-627a-4a7b-93a3-9cc3dce48661 admin admin] RBAC: Authorizing `identity:delete_project(project_id=194a16f2ffd446b3b3e9742e45fd1934)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:49.557095 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731d20ed-627a-4a7b-93a3-9cc3dce48661 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:49.557213 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c0e52d-db7b-44bd-b889-a1cd67e8e6c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8c9fce785074a5fbf2b6b0efe538c59, user_id=f0c0d784aea54218bfec1485c1d79c6b, role_id=b8c98fddc0fd4436b577fd1d3361bc38)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:49.560873 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c0e52d-db7b-44bd-b889-a1cd67e8e6c8 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:49.578915 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1158/2286] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 15:38:49 2022] PUT /identity/v3/projects/a8c9fce785074a5fbf2b6b0efe538c59/users/f0c0d784aea54218bfec1485c1d79c6b/roles/b8c98fddc0fd4436b577fd1d3361bc38 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:38:49.590337 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6011e8dc-68e4-4bba-ae8f-c8ca97396b2b None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:49.590640 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6011e8dc-68e4-4bba-ae8f-c8ca97396b2b None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:49.590922 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6011e8dc-68e4-4bba-ae8f-c8ca97396b2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:49.612880 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:49.619768 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-731d20ed-627a-4a7b-93a3-9cc3dce48661 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '194a16f2ffd446b3b3e9742e45fd1934', 'request_id': 'req-731d20ed-627a-4a7b-93a3-9cc3dce48661'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:49.623315 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=91,keystone:DELETE=6,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:49.627499 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-6011e8dc-68e4-4bba-ae8f-c8ca97396b2b None None] MFA Rules not processed for user `f0c0d784aea54218bfec1485c1d79c6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:38:49.628783 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-6011e8dc-68e4-4bba-ae8f-c8ca97396b2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:49.630357 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-731d20ed-627a-4a7b-93a3-9cc3dce48661 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '194a16f2ffd446b3b3e9742e45fd1934', 'request_id': 'req-731d20ed-627a-4a7b-93a3-9cc3dce48661'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:49.639860 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-731d20ed-627a-4a7b-93a3-9cc3dce48661 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '194a16f2ffd446b3b3e9742e45fd1934', 'request_id': 'req-731d20ed-627a-4a7b-93a3-9cc3dce48661'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:49.656037 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-731d20ed-627a-4a7b-93a3-9cc3dce48661 admin admin] The token cache is being invalidate because project 194a16f2ffd446b3b3e9742e45fd1934 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 15:38:49.656406 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-731d20ed-627a-4a7b-93a3-9cc3dce48661 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:49.657220 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-731d20ed-627a-4a7b-93a3-9cc3dce48661 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:49.662742 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1129/2287] 10.222.0.99 () {58 vars in 1232 bytes} [Thu Jul 14 15:38:49 2022] DELETE /identity/v3/projects/194a16f2ffd446b3b3e9742e45fd1934 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:38:49.664250 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731d20ed-627a-4a7b-93a3-9cc3dce48661 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:49.665629 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-731d20ed-627a-4a7b-93a3-9cc3dce48661 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:49.721720 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1159/2288] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:38:49 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 15:38:49.725943 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6011e8dc-68e4-4bba-ae8f-c8ca97396b2b None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:49.727286 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-6011e8dc-68e4-4bba-ae8f-c8ca97396b2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:49.750636 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731d20ed-627a-4a7b-93a3-9cc3dce48661 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:49.756058 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6e9cd6-4a23-4cc4-b7de-1726b915cd8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:49.756992 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6e9cd6-4a23-4cc4-b7de-1726b915cd8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:49.757311 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6e9cd6-4a23-4cc4-b7de-1726b915cd8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:49.757554 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6e9cd6-4a23-4cc4-b7de-1726b915cd8e admin admin] PATH_INFO: `/v3/users/b81bb209bd02495b8e66b2dc858fd3ab` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:49.758796 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6e9cd6-4a23-4cc4-b7de-1726b915cd8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=b81bb209bd02495b8e66b2dc858fd3ab)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:49.760833 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6e9cd6-4a23-4cc4-b7de-1726b915cd8e admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:49.812962 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6011e8dc-68e4-4bba-ae8f-c8ca97396b2b None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:49.818811 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682b03c8-ec0a-44f6-8d2b-53375bafe158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:49.819501 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682b03c8-ec0a-44f6-8d2b-53375bafe158 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:49.819872 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682b03c8-ec0a-44f6-8d2b-53375bafe158 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:49.820159 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682b03c8-ec0a-44f6-8d2b-53375bafe158 admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:49.820929 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682b03c8-ec0a-44f6-8d2b-53375bafe158 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:49.823013 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682b03c8-ec0a-44f6-8d2b-53375bafe158 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:49.842203 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-cd6e9cd6-4a23-4cc4-b7de-1726b915cd8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b81bb209bd02495b8e66b2dc858fd3ab'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:49.865928 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-cd6e9cd6-4a23-4cc4-b7de-1726b915cd8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b81bb209bd02495b8e66b2dc858fd3ab'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:49.887229 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1160/2289] 10.222.0.99 () {60 vars in 1152 bytes} [Thu Jul 14 15:38:49 2022] POST /identity/v3/projects => generated 375 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 15:38:49.889416 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682b03c8-ec0a-44f6-8d2b-53375bafe158 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:49.912591 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-cd6e9cd6-4a23-4cc4-b7de-1726b915cd8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b81bb209bd02495b8e66b2dc858fd3ab'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:49.915968 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-cd6e9cd6-4a23-4cc4-b7de-1726b915cd8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b81bb209bd02495b8e66b2dc858fd3ab'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:49.917969 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-cd6e9cd6-4a23-4cc4-b7de-1726b915cd8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b81bb209bd02495b8e66b2dc858fd3ab'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:49.957845 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1130/2290] 10.222.0.99 () {58 vars in 1223 bytes} [Thu Jul 14 15:38:49 2022] DELETE /identity/v3/users/b81bb209bd02495b8e66b2dc858fd3ab => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 15:38:50.046222 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682b03c8-ec0a-44f6-8d2b-53375bafe158 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:50.082211 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:50.125720 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89d2409-9344-4f67-8b47-0de9a3704404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:50.127030 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89d2409-9344-4f67-8b47-0de9a3704404 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:50.127325 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89d2409-9344-4f67-8b47-0de9a3704404 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:50.127680 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89d2409-9344-4f67-8b47-0de9a3704404 admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:50.128381 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89d2409-9344-4f67-8b47-0de9a3704404 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:50.130357 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89d2409-9344-4f67-8b47-0de9a3704404 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:50.154426 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1161/2291] 10.222.0.99 () {60 vars in 1143 bytes} [Thu Jul 14 15:38:49 2022] POST /identity/v3/users => generated 294 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:38:50.159575 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6e9cd6-4a23-4cc4-b7de-1726b915cd8e admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:50.161579 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-cd6e9cd6-4a23-4cc4-b7de-1726b915cd8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:50.175954 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6e9cd6-4a23-4cc4-b7de-1726b915cd8e admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:50.181214 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead2b0df-7398-4c37-a737-f6b74370c386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:50.181913 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead2b0df-7398-4c37-a737-f6b74370c386 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:50.182200 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead2b0df-7398-4c37-a737-f6b74370c386 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:50.182480 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead2b0df-7398-4c37-a737-f6b74370c386 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:50.183255 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead2b0df-7398-4c37-a737-f6b74370c386 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:50.184957 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead2b0df-7398-4c37-a737-f6b74370c386 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:50.199658 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1131/2292] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:38:50 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:38:50.201383 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89d2409-9344-4f67-8b47-0de9a3704404 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:50.215514 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89d2409-9344-4f67-8b47-0de9a3704404 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:50.221934 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f092000-71c8-48bb-adbc-3d918550a332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:50.222845 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f092000-71c8-48bb-adbc-3d918550a332 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:50.223135 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f092000-71c8-48bb-adbc-3d918550a332 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:50.223419 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f092000-71c8-48bb-adbc-3d918550a332 admin admin] PATH_INFO: `/v3/projects/c4cf2c34fdaa44c6af101ac2014f7181/users/ca2758ea0d454213b683af7ffae84ec9/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:50.246101 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f092000-71c8-48bb-adbc-3d918550a332 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4cf2c34fdaa44c6af101ac2014f7181, user_id=ca2758ea0d454213b683af7ffae84ec9, role_id=d0f6ba1314f14d9f9c816bdb5d0c3aac)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:50.249194 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f092000-71c8-48bb-adbc-3d918550a332 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:50.264276 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1162/2293] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 15:38:50 2022] PUT /identity/v3/projects/c4cf2c34fdaa44c6af101ac2014f7181/users/ca2758ea0d454213b683af7ffae84ec9/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:38:50.268886 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead2b0df-7398-4c37-a737-f6b74370c386 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:50.279232 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead2b0df-7398-4c37-a737-f6b74370c386 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:50.330242 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:50.346834 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5d0af2-d6f3-4a46-afb4-6c19953a7992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:50.347319 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5d0af2-d6f3-4a46-afb4-6c19953a7992 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:50.347656 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5d0af2-d6f3-4a46-afb4-6c19953a7992 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:50.347940 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5d0af2-d6f3-4a46-afb4-6c19953a7992 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:50.348673 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5d0af2-d6f3-4a46-afb4-6c19953a7992 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:50.350323 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5d0af2-d6f3-4a46-afb4-6c19953a7992 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:50.362491 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1132/2294] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:38:50 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 15:38:50.366859 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f092000-71c8-48bb-adbc-3d918550a332 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:50.378230 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f092000-71c8-48bb-adbc-3d918550a332 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:50.382988 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2ad806-ed33-47a6-99e2-4629b7874d61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:50.383915 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2ad806-ed33-47a6-99e2-4629b7874d61 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:50.384203 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2ad806-ed33-47a6-99e2-4629b7874d61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:50.384482 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2ad806-ed33-47a6-99e2-4629b7874d61 admin admin] PATH_INFO: `/v3/projects/c4cf2c34fdaa44c6af101ac2014f7181/users/ca2758ea0d454213b683af7ffae84ec9/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:50.386466 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2ad806-ed33-47a6-99e2-4629b7874d61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4cf2c34fdaa44c6af101ac2014f7181, user_id=ca2758ea0d454213b683af7ffae84ec9, role_id=d0f6ba1314f14d9f9c816bdb5d0c3aac)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:50.389148 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2ad806-ed33-47a6-99e2-4629b7874d61 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:50.400918 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1163/2295] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 15:38:50 2022] PUT /identity/v3/projects/c4cf2c34fdaa44c6af101ac2014f7181/users/ca2758ea0d454213b683af7ffae84ec9/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:38:50.411451 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb34f904-8534-49df-8f83-8ae51287e1d3 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:50.411788 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb34f904-8534-49df-8f83-8ae51287e1d3 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:50.412088 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb34f904-8534-49df-8f83-8ae51287e1d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:50.448273 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-bb34f904-8534-49df-8f83-8ae51287e1d3 None None] MFA Rules not processed for user `ca2758ea0d454213b683af7ffae84ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:38:50.449719 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-bb34f904-8534-49df-8f83-8ae51287e1d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:50.467475 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2ad806-ed33-47a6-99e2-4629b7874d61 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:50.469447 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-8f2ad806-ed33-47a6-99e2-4629b7874d61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:50.552464 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1133/2296] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:38:50 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:38:50.553145 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2ad806-ed33-47a6-99e2-4629b7874d61 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:50.558720 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b3c76c-9920-4df7-adc3-964fad8bf1bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:50.559486 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b3c76c-9920-4df7-adc3-964fad8bf1bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:50.560203 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b3c76c-9920-4df7-adc3-964fad8bf1bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:50.560203 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b3c76c-9920-4df7-adc3-964fad8bf1bc admin admin] PATH_INFO: `/v3/projects/b95683a167804b6289ed527db3fc8142` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:50.561234 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b3c76c-9920-4df7-adc3-964fad8bf1bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=b95683a167804b6289ed527db3fc8142)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:50.563157 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b3c76c-9920-4df7-adc3-964fad8bf1bc admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:50.564867 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb34f904-8534-49df-8f83-8ae51287e1d3 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:50.566375 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-bb34f904-8534-49df-8f83-8ae51287e1d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:50.625209 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-96b3c76c-9920-4df7-adc3-964fad8bf1bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b95683a167804b6289ed527db3fc8142', 'request_id': 'req-96b3c76c-9920-4df7-adc3-964fad8bf1bc'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:50.634258 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-96b3c76c-9920-4df7-adc3-964fad8bf1bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b95683a167804b6289ed527db3fc8142', 'request_id': 'req-96b3c76c-9920-4df7-adc3-964fad8bf1bc'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:50.641011 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-96b3c76c-9920-4df7-adc3-964fad8bf1bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b95683a167804b6289ed527db3fc8142', 'request_id': 'req-96b3c76c-9920-4df7-adc3-964fad8bf1bc'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:50.647155 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb34f904-8534-49df-8f83-8ae51287e1d3 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:50.656471 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-96b3c76c-9920-4df7-adc3-964fad8bf1bc admin admin] The token cache is being invalidate because project b95683a167804b6289ed527db3fc8142 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 15:38:50.657139 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-96b3c76c-9920-4df7-adc3-964fad8bf1bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:50.658380 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-96b3c76c-9920-4df7-adc3-964fad8bf1bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:50.663396 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1164/2297] 10.222.0.99 () {58 vars in 1232 bytes} [Thu Jul 14 15:38:50 2022] DELETE /identity/v3/projects/b95683a167804b6289ed527db3fc8142 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 15:38:50.665402 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7bf941-8eb3-4625-b4ed-2cd1dc7a2b76 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:50.665629 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7bf941-8eb3-4625-b4ed-2cd1dc7a2b76 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:50.665995 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7bf941-8eb3-4625-b4ed-2cd1dc7a2b76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:50.690539 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-5b7bf941-8eb3-4625-b4ed-2cd1dc7a2b76 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:38:50.691790 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-5b7bf941-8eb3-4625-b4ed-2cd1dc7a2b76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:50.702450 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:50.703515 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9161d96d-41a7-46c9-952e-3ee796048e16 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3e1a2b83f3d4d88ba5084dfaffd22f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:50.704294 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9161d96d-41a7-46c9-952e-3ee796048e16 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:50.704711 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9161d96d-41a7-46c9-952e-3ee796048e16 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:50.705070 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9161d96d-41a7-46c9-952e-3ee796048e16 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:50.707131 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-9161d96d-41a7-46c9-952e-3ee796048e16 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:50.750083 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:50.757691 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1165/2298] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:38:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 15:38:50.765659 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df31fa4-18c7-4433-8533-ae9ddfad0ca5 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:50.765935 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df31fa4-18c7-4433-8533-ae9ddfad0ca5 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:50.766203 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df31fa4-18c7-4433-8533-ae9ddfad0ca5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:50.769630 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9161d96d-41a7-46c9-952e-3ee796048e16 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:50.771157 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9161d96d-41a7-46c9-952e-3ee796048e16 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:50.781797 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1134/2299] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:38:50 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:38:50.788459 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-8df31fa4-18c7-4433-8533-ae9ddfad0ca5 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:38:50.789665 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-8df31fa4-18c7-4433-8533-ae9ddfad0ca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:50.798058 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1166/2300] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:38:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 15:38:50.802627 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9161d96d-41a7-46c9-952e-3ee796048e16 service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:50.813313 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9161d96d-41a7-46c9-952e-3ee796048e16 service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:50.819126 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc16b802-860f-4d8b-920a-428441475df3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:50.819853 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc16b802-860f-4d8b-920a-428441475df3 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:50.820148 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc16b802-860f-4d8b-920a-428441475df3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:50.820427 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc16b802-860f-4d8b-920a-428441475df3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:50.821159 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc16b802-860f-4d8b-920a-428441475df3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:38:50.821475 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc16b802-860f-4d8b-920a-428441475df3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:50.823069 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc16b802-860f-4d8b-920a-428441475df3 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:50.835205 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1135/2301] 10.222.0.99 () {58 vars in 1152 bytes} [Thu Jul 14 15:38:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:38:50.839773 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df31fa4-18c7-4433-8533-ae9ddfad0ca5 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:50.849974 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df31fa4-18c7-4433-8533-ae9ddfad0ca5 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:50.854607 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e1d693-688d-4af6-a0bd-8b0127aea259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:50.855256 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e1d693-688d-4af6-a0bd-8b0127aea259 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:50.855541 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e1d693-688d-4af6-a0bd-8b0127aea259 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:50.855850 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e1d693-688d-4af6-a0bd-8b0127aea259 admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:50.856566 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e1d693-688d-4af6-a0bd-8b0127aea259 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:50.858479 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e1d693-688d-4af6-a0bd-8b0127aea259 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:50.885839 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1167/2302] 10.222.0.99 () {60 vars in 1152 bytes} [Thu Jul 14 15:38:50 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 15:38:50.890289 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc16b802-860f-4d8b-920a-428441475df3 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:50.901635 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc16b802-860f-4d8b-920a-428441475df3 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:50.970432 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d133795-d818-4bd6-ac4e-9215005976a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:50.971421 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d133795-d818-4bd6-ac4e-9215005976a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:50.971753 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d133795-d818-4bd6-ac4e-9215005976a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:50.972052 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d133795-d818-4bd6-ac4e-9215005976a9 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:50.972667 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d133795-d818-4bd6-ac4e-9215005976a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:50.974753 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d133795-d818-4bd6-ac4e-9215005976a9 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:51.005414 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1136/2303] 10.222.0.99 () {60 vars in 1143 bytes} [Thu Jul 14 15:38:50 2022] POST /identity/v3/users => generated 286 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 15:38:51.010200 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e1d693-688d-4af6-a0bd-8b0127aea259 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:51.022903 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e1d693-688d-4af6-a0bd-8b0127aea259 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:51.027635 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3664becf-1fb5-4e52-bf87-853ed6ae4c10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:51.028326 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3664becf-1fb5-4e52-bf87-853ed6ae4c10 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:51.028606 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3664becf-1fb5-4e52-bf87-853ed6ae4c10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:51.028886 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3664becf-1fb5-4e52-bf87-853ed6ae4c10 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:51.029678 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3664becf-1fb5-4e52-bf87-853ed6ae4c10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:51.031436 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3664becf-1fb5-4e52-bf87-853ed6ae4c10 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:51.051361 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1168/2304] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:38:51 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 15:38:51.052559 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d133795-d818-4bd6-ac4e-9215005976a9 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:51.064941 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d133795-d818-4bd6-ac4e-9215005976a9 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:51.069924 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c953979-40ce-48b1-99b1-8e3c014e61d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:51.070795 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c953979-40ce-48b1-99b1-8e3c014e61d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:51.071087 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c953979-40ce-48b1-99b1-8e3c014e61d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:51.071501 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c953979-40ce-48b1-99b1-8e3c014e61d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:51.072318 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c953979-40ce-48b1-99b1-8e3c014e61d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:51.073914 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c953979-40ce-48b1-99b1-8e3c014e61d4 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:51.087152 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1137/2305] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:38:51 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 15:38:51.090868 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3664becf-1fb5-4e52-bf87-853ed6ae4c10 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:51.112934 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3664becf-1fb5-4e52-bf87-853ed6ae4c10 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:51.118762 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ec85ea-c5da-457c-8e89-e65baa9eea03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:51.119655 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ec85ea-c5da-457c-8e89-e65baa9eea03 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:51.119966 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ec85ea-c5da-457c-8e89-e65baa9eea03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:51.120223 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ec85ea-c5da-457c-8e89-e65baa9eea03 admin admin] PATH_INFO: `/v3/projects/2345d49f9fee4842826a396c43132a33/users/1f3411c8a7454203a4fbd748f1f3031a/roles/b8c98fddc0fd4436b577fd1d3361bc38` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:51.138369 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ec85ea-c5da-457c-8e89-e65baa9eea03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2345d49f9fee4842826a396c43132a33, user_id=1f3411c8a7454203a4fbd748f1f3031a, role_id=b8c98fddc0fd4436b577fd1d3361bc38)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:51.141240 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ec85ea-c5da-457c-8e89-e65baa9eea03 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:51.160132 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1169/2306] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 15:38:51 2022] PUT /identity/v3/projects/2345d49f9fee4842826a396c43132a33/users/1f3411c8a7454203a4fbd748f1f3031a/roles/b8c98fddc0fd4436b577fd1d3361bc38 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 15:38:51.165198 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570445e9-4871-43ec-b072-1f3f0eacb48b None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:51.165660 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570445e9-4871-43ec-b072-1f3f0eacb48b None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:51.166005 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570445e9-4871-43ec-b072-1f3f0eacb48b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:51.185113 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-570445e9-4871-43ec-b072-1f3f0eacb48b None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:38:51.186465 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-570445e9-4871-43ec-b072-1f3f0eacb48b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:51.257616 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1138/2307] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:38:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:38:51.264055 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ec85ea-c5da-457c-8e89-e65baa9eea03 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:51.265942 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-38ec85ea-c5da-457c-8e89-e65baa9eea03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:51.328984 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ec85ea-c5da-457c-8e89-e65baa9eea03 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:51.333403 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbab024-6711-4e8d-b506-dc04439ba6f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:51.334071 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbab024-6711-4e8d-b506-dc04439ba6f0 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:51.334351 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbab024-6711-4e8d-b506-dc04439ba6f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:51.334634 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbab024-6711-4e8d-b506-dc04439ba6f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:51.344574 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbab024-6711-4e8d-b506-dc04439ba6f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:51.347063 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbab024-6711-4e8d-b506-dc04439ba6f0 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:51.357018 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1170/2308] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:38:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:38:52.066301 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93269fdc-dfa6-4de0-8b6a-f42c64c45a84 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:52.067147 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93269fdc-dfa6-4de0-8b6a-f42c64c45a84 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:52.067736 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93269fdc-dfa6-4de0-8b6a-f42c64c45a84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:52.096577 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-93269fdc-dfa6-4de0-8b6a-f42c64c45a84 None None] MFA Rules not processed for user `1f3411c8a7454203a4fbd748f1f3031a`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:38:52.098025 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-93269fdc-dfa6-4de0-8b6a-f42c64c45a84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:52.136200 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:52.181177 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1139/2309] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:38:52 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 15:38:52.192523 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbab024-6711-4e8d-b506-dc04439ba6f0 service neutron] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:52.194070 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-4fbab024-6711-4e8d-b506-dc04439ba6f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:52.283000 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbab024-6711-4e8d-b506-dc04439ba6f0 service neutron] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:52.286953 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f26de4-860d-4687-a025-d8c5b7a2cc32 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3e1a2b83f3d4d88ba5084dfaffd22f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:52.287628 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f26de4-860d-4687-a025-d8c5b7a2cc32 service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:52.287928 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f26de4-860d-4687-a025-d8c5b7a2cc32 service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:52.288212 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f26de4-860d-4687-a025-d8c5b7a2cc32 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:52.297333 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f26de4-860d-4687-a025-d8c5b7a2cc32 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:52.299422 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f26de4-860d-4687-a025-d8c5b7a2cc32 service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:52.303387 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:52.310520 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1171/2310] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:38:52 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:38:55.806203 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91a4999-2870-47e8-9a3d-8da78f8a51a9 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:55.806870 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91a4999-2870-47e8-9a3d-8da78f8a51a9 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:55.806870 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91a4999-2870-47e8-9a3d-8da78f8a51a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:55.827839 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-d91a4999-2870-47e8-9a3d-8da78f8a51a9 None None] MFA Rules not processed for user `1f3411c8a7454203a4fbd748f1f3031a`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:38:55.829224 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d91a4999-2870-47e8-9a3d-8da78f8a51a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:55.836318 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1140/2311] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:38:55 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 15:38:56.499998 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f26de4-860d-4687-a025-d8c5b7a2cc32 service cinder] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:56.502252 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-e9f26de4-860d-4687-a025-d8c5b7a2cc32 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:56.521041 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f26de4-860d-4687-a025-d8c5b7a2cc32 service cinder] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:56.529803 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd26936-f0f8-4423-9561-974bdd2b967c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:56.531340 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd26936-f0f8-4423-9561-974bdd2b967c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:56.531969 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd26936-f0f8-4423-9561-974bdd2b967c admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:56.532531 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd26936-f0f8-4423-9561-974bdd2b967c admin admin] PATH_INFO: `/v3/users/f0c0d784aea54218bfec1485c1d79c6b` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:56.534647 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd26936-f0f8-4423-9561-974bdd2b967c admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0c0d784aea54218bfec1485c1d79c6b)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:56.537895 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd26936-f0f8-4423-9561-974bdd2b967c admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:56.616838 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-bcd26936-f0f8-4423-9561-974bdd2b967c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0c0d784aea54218bfec1485c1d79c6b'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:56.625070 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-bcd26936-f0f8-4423-9561-974bdd2b967c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0c0d784aea54218bfec1485c1d79c6b'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:56.645328 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-bcd26936-f0f8-4423-9561-974bdd2b967c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0c0d784aea54218bfec1485c1d79c6b'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:56.646025 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-bcd26936-f0f8-4423-9561-974bdd2b967c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0c0d784aea54218bfec1485c1d79c6b'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:56.646631 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-bcd26936-f0f8-4423-9561-974bdd2b967c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0c0d784aea54218bfec1485c1d79c6b'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:56.660217 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1172/2312] 10.222.0.99 () {58 vars in 1223 bytes} [Thu Jul 14 15:38:56 2022] DELETE /identity/v3/users/f0c0d784aea54218bfec1485c1d79c6b => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:38:56.665205 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079296ee-4102-4eac-ad55-b88cdd610320 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:56.665556 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079296ee-4102-4eac-ad55-b88cdd610320 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:56.665851 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079296ee-4102-4eac-ad55-b88cdd610320 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:56.690102 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-079296ee-4102-4eac-ad55-b88cdd610320 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:38:56.691482 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-079296ee-4102-4eac-ad55-b88cdd610320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:56.769176 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1141/2313] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:38:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 15:38:56.779502 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd26936-f0f8-4423-9561-974bdd2b967c admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:56.781801 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-bcd26936-f0f8-4423-9561-974bdd2b967c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:56.865158 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd26936-f0f8-4423-9561-974bdd2b967c admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:56.869082 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f6ee8c-12ec-4603-8f53-40a6ab505c58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:56.869625 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f6ee8c-12ec-4603-8f53-40a6ab505c58 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:56.869918 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f6ee8c-12ec-4603-8f53-40a6ab505c58 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:56.870247 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f6ee8c-12ec-4603-8f53-40a6ab505c58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:56.880616 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f6ee8c-12ec-4603-8f53-40a6ab505c58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:56.882095 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f6ee8c-12ec-4603-8f53-40a6ab505c58 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:56.892831 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1173/2314] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:38:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:38:57.346570 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079296ee-4102-4eac-ad55-b88cdd610320 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:57.348590 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-079296ee-4102-4eac-ad55-b88cdd610320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:57.361397 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079296ee-4102-4eac-ad55-b88cdd610320 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:57.366894 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713d1859-dccb-45c3-9b8e-f04c890d70c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:57.367837 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713d1859-dccb-45c3-9b8e-f04c890d70c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:57.368316 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713d1859-dccb-45c3-9b8e-f04c890d70c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:57.368602 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713d1859-dccb-45c3-9b8e-f04c890d70c0 admin admin] PATH_INFO: `/v3/projects/a8c9fce785074a5fbf2b6b0efe538c59` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:57.369819 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713d1859-dccb-45c3-9b8e-f04c890d70c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8c9fce785074a5fbf2b6b0efe538c59)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:57.371792 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713d1859-dccb-45c3-9b8e-f04c890d70c0 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:57.422581 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-713d1859-dccb-45c3-9b8e-f04c890d70c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8c9fce785074a5fbf2b6b0efe538c59', 'request_id': 'req-713d1859-dccb-45c3-9b8e-f04c890d70c0'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:57.430135 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-713d1859-dccb-45c3-9b8e-f04c890d70c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8c9fce785074a5fbf2b6b0efe538c59', 'request_id': 'req-713d1859-dccb-45c3-9b8e-f04c890d70c0'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:57.436129 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-713d1859-dccb-45c3-9b8e-f04c890d70c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8c9fce785074a5fbf2b6b0efe538c59', 'request_id': 'req-713d1859-dccb-45c3-9b8e-f04c890d70c0'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:57.449324 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-713d1859-dccb-45c3-9b8e-f04c890d70c0 admin admin] The token cache is being invalidate because project a8c9fce785074a5fbf2b6b0efe538c59 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 15:38:57.449640 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-713d1859-dccb-45c3-9b8e-f04c890d70c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:57.450251 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-713d1859-dccb-45c3-9b8e-f04c890d70c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:57.451943 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1142/2315] 10.222.0.99 () {58 vars in 1232 bytes} [Thu Jul 14 15:38:57 2022] DELETE /identity/v3/projects/a8c9fce785074a5fbf2b6b0efe538c59 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:38:57.457288 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f6ee8c-12ec-4603-8f53-40a6ab505c58 service neutron] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:57.468109 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f6ee8c-12ec-4603-8f53-40a6ab505c58 service neutron] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:57.535635 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e79af8f-8d59-49ef-b4f1-c8e3807fd2de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:57.537161 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e79af8f-8d59-49ef-b4f1-c8e3807fd2de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:57.537455 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e79af8f-8d59-49ef-b4f1-c8e3807fd2de admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:57.537739 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e79af8f-8d59-49ef-b4f1-c8e3807fd2de admin admin] PATH_INFO: `/v3/users/ca2758ea0d454213b683af7ffae84ec9` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:57.539144 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e79af8f-8d59-49ef-b4f1-c8e3807fd2de admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca2758ea0d454213b683af7ffae84ec9)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:57.541142 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e79af8f-8d59-49ef-b4f1-c8e3807fd2de admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:57.562993 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:57.606257 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-0e79af8f-8d59-49ef-b4f1-c8e3807fd2de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca2758ea0d454213b683af7ffae84ec9'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:57.614657 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-0e79af8f-8d59-49ef-b4f1-c8e3807fd2de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca2758ea0d454213b683af7ffae84ec9'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:57.628421 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-0e79af8f-8d59-49ef-b4f1-c8e3807fd2de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca2758ea0d454213b683af7ffae84ec9'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:57.629118 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-0e79af8f-8d59-49ef-b4f1-c8e3807fd2de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca2758ea0d454213b683af7ffae84ec9'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:57.629717 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-0e79af8f-8d59-49ef-b4f1-c8e3807fd2de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca2758ea0d454213b683af7ffae84ec9'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:57.641964 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1174/2316] 10.222.0.99 () {58 vars in 1223 bytes} [Thu Jul 14 15:38:57 2022] DELETE /identity/v3/users/ca2758ea0d454213b683af7ffae84ec9 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:38:57.961237 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713d1859-dccb-45c3-9b8e-f04c890d70c0 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:57.963184 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-713d1859-dccb-45c3-9b8e-f04c890d70c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:57.983844 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713d1859-dccb-45c3-9b8e-f04c890d70c0 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:58.060333 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:58.070997 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b76c52f-aae1-4d95-95a8-5a91c23c64f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:58.072114 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b76c52f-aae1-4d95-95a8-5a91c23c64f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:58.072555 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b76c52f-aae1-4d95-95a8-5a91c23c64f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:58.072866 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b76c52f-aae1-4d95-95a8-5a91c23c64f0 admin admin] PATH_INFO: `/v3/projects/c4cf2c34fdaa44c6af101ac2014f7181` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:58.074049 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b76c52f-aae1-4d95-95a8-5a91c23c64f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4cf2c34fdaa44c6af101ac2014f7181)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:58.076434 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b76c52f-aae1-4d95-95a8-5a91c23c64f0 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:58.141694 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2b76c52f-aae1-4d95-95a8-5a91c23c64f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4cf2c34fdaa44c6af101ac2014f7181', 'request_id': 'req-2b76c52f-aae1-4d95-95a8-5a91c23c64f0'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:58.148458 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2b76c52f-aae1-4d95-95a8-5a91c23c64f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4cf2c34fdaa44c6af101ac2014f7181', 'request_id': 'req-2b76c52f-aae1-4d95-95a8-5a91c23c64f0'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:58.154147 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2b76c52f-aae1-4d95-95a8-5a91c23c64f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4cf2c34fdaa44c6af101ac2014f7181', 'request_id': 'req-2b76c52f-aae1-4d95-95a8-5a91c23c64f0'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:58.168862 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2b76c52f-aae1-4d95-95a8-5a91c23c64f0 admin admin] The token cache is being invalidate because project c4cf2c34fdaa44c6af101ac2014f7181 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 15:38:58.169157 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2b76c52f-aae1-4d95-95a8-5a91c23c64f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:58.169825 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2b76c52f-aae1-4d95-95a8-5a91c23c64f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:38:58.171651 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1143/2317] 10.222.0.99 () {58 vars in 1232 bytes} [Thu Jul 14 15:38:57 2022] DELETE /identity/v3/projects/c4cf2c34fdaa44c6af101ac2014f7181 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:38:58.177498 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0c9ee7-57b2-4a64-afde-04d88fad52ab None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:58.177801 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0c9ee7-57b2-4a64-afde-04d88fad52ab None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:58.178090 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0c9ee7-57b2-4a64-afde-04d88fad52ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:58.200435 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-fb0c9ee7-57b2-4a64-afde-04d88fad52ab None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:38:58.201730 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-fb0c9ee7-57b2-4a64-afde-04d88fad52ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:58.275351 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1175/2318] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:38:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:38:58.277087 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2be1d6-d8bd-4d41-896f-6442a43af02c None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:58.277385 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2be1d6-d8bd-4d41-896f-6442a43af02c None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:58.277674 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2be1d6-d8bd-4d41-896f-6442a43af02c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:58.297258 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-de2be1d6-d8bd-4d41-896f-6442a43af02c None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:38:58.299012 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-de2be1d6-d8bd-4d41-896f-6442a43af02c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:58.307121 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1144/2319] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:38:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:38:58.311341 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0c9ee7-57b2-4a64-afde-04d88fad52ab None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:58.321615 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0c9ee7-57b2-4a64-afde-04d88fad52ab None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:58.326803 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f5369d-8684-4cd6-9825-99d1b275ba12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:58.327253 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f5369d-8684-4cd6-9825-99d1b275ba12 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:58.327540 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f5369d-8684-4cd6-9825-99d1b275ba12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:58.327855 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f5369d-8684-4cd6-9825-99d1b275ba12 admin admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:58.328598 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f5369d-8684-4cd6-9825-99d1b275ba12 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:38:58.328944 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f5369d-8684-4cd6-9825-99d1b275ba12 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:58.331265 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f5369d-8684-4cd6-9825-99d1b275ba12 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:58.343086 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1176/2320] 10.222.0.99 () {58 vars in 1152 bytes} [Thu Jul 14 15:38:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:38:58.347850 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2be1d6-d8bd-4d41-896f-6442a43af02c None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:58.357266 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2be1d6-d8bd-4d41-896f-6442a43af02c None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:58.362772 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6b02d4-2f90-4f1f-aef3-66f50139e4e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:58.363438 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6b02d4-2f90-4f1f-aef3-66f50139e4e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:58.364702 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6b02d4-2f90-4f1f-aef3-66f50139e4e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:58.364973 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6b02d4-2f90-4f1f-aef3-66f50139e4e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:58.365803 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6b02d4-2f90-4f1f-aef3-66f50139e4e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:58.367687 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6b02d4-2f90-4f1f-aef3-66f50139e4e0 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:58.401261 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1145/2321] 10.222.0.99 () {60 vars in 1152 bytes} [Thu Jul 14 15:38:58 2022] POST /identity/v3/projects => generated 379 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:38:58.402329 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f5369d-8684-4cd6-9825-99d1b275ba12 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:58.413160 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f5369d-8684-4cd6-9825-99d1b275ba12 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:58.479245 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6700e6-dd13-448e-8b22-81616a75cb96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:58.480791 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6700e6-dd13-448e-8b22-81616a75cb96 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:58.481000 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6700e6-dd13-448e-8b22-81616a75cb96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:58.481587 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6700e6-dd13-448e-8b22-81616a75cb96 admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:58.482327 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6700e6-dd13-448e-8b22-81616a75cb96 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:58.484304 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6700e6-dd13-448e-8b22-81616a75cb96 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:58.514535 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1177/2322] 10.222.0.99 () {60 vars in 1143 bytes} [Thu Jul 14 15:38:58 2022] POST /identity/v3/users => generated 290 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:38:58.515847 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6b02d4-2f90-4f1f-aef3-66f50139e4e0 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:58.528510 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6b02d4-2f90-4f1f-aef3-66f50139e4e0 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:58.535313 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e2a7e9-69af-481d-9c62-226df05d72e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:58.536100 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e2a7e9-69af-481d-9c62-226df05d72e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:58.536394 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e2a7e9-69af-481d-9c62-226df05d72e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:58.536676 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e2a7e9-69af-481d-9c62-226df05d72e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:58.538041 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e2a7e9-69af-481d-9c62-226df05d72e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:58.539673 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e2a7e9-69af-481d-9c62-226df05d72e2 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:58.551824 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1146/2323] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:38:58 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 15:38:58.556024 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6700e6-dd13-448e-8b22-81616a75cb96 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:58.568375 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6700e6-dd13-448e-8b22-81616a75cb96 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:58.573172 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e445143-1cd0-4639-acea-52b45dc44723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:58.573841 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e445143-1cd0-4639-acea-52b45dc44723 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:58.574307 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e445143-1cd0-4639-acea-52b45dc44723 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:58.574755 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e445143-1cd0-4639-acea-52b45dc44723 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:58.575490 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e445143-1cd0-4639-acea-52b45dc44723 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:58.577265 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e445143-1cd0-4639-acea-52b45dc44723 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:58.580364 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:58.590640 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1178/2324] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:38:58 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:38:58.596575 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e2a7e9-69af-481d-9c62-226df05d72e2 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:58.609028 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e2a7e9-69af-481d-9c62-226df05d72e2 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:58.621071 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46303d3-b7ae-479f-9884-6e51eb71f209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:58.622316 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46303d3-b7ae-479f-9884-6e51eb71f209 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:58.622831 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46303d3-b7ae-479f-9884-6e51eb71f209 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:58.623851 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46303d3-b7ae-479f-9884-6e51eb71f209 admin admin] PATH_INFO: `/v3/projects/599a768f6b9d4134a4b8e05a7e5f5e87/users/a0e4900b6ad24eb19fe169d9c8d8c3a7/roles/b8c98fddc0fd4436b577fd1d3361bc38` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:58.648747 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46303d3-b7ae-479f-9884-6e51eb71f209 admin admin] RBAC: Authorizing `identity:create_grant(project_id=599a768f6b9d4134a4b8e05a7e5f5e87, user_id=a0e4900b6ad24eb19fe169d9c8d8c3a7, role_id=b8c98fddc0fd4436b577fd1d3361bc38)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:58.653113 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46303d3-b7ae-479f-9884-6e51eb71f209 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:58.680284 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1147/2325] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 15:38:58 2022] PUT /identity/v3/projects/599a768f6b9d4134a4b8e05a7e5f5e87/users/a0e4900b6ad24eb19fe169d9c8d8c3a7/roles/b8c98fddc0fd4436b577fd1d3361bc38 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:38:58.685235 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616ad3b7-cb21-4d9c-9dfa-9b173579a81f None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:58.685619 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616ad3b7-cb21-4d9c-9dfa-9b173579a81f None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:58.685972 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616ad3b7-cb21-4d9c-9dfa-9b173579a81f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:58.712202 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-616ad3b7-cb21-4d9c-9dfa-9b173579a81f None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:38:58.713269 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-616ad3b7-cb21-4d9c-9dfa-9b173579a81f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:58.798290 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1179/2326] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:38:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 15:38:58.809332 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46303d3-b7ae-479f-9884-6e51eb71f209 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:58.811327 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e46303d3-b7ae-479f-9884-6e51eb71f209 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:58.912304 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46303d3-b7ae-479f-9884-6e51eb71f209 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:58.920399 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5c4353-7a5e-470a-83c9-f3bf8fb80082 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:58.923610 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5c4353-7a5e-470a-83c9-f3bf8fb80082 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:58.923997 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5c4353-7a5e-470a-83c9-f3bf8fb80082 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:58.924287 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5c4353-7a5e-470a-83c9-f3bf8fb80082 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:58.942719 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5c4353-7a5e-470a-83c9-f3bf8fb80082 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:58.946373 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5c4353-7a5e-470a-83c9-f3bf8fb80082 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:58.961065 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1148/2327] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:38:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:39:04.503553 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9724ab-50c0-4a72-a751-065b2d71926a None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:39:04.504228 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9724ab-50c0-4a72-a751-065b2d71926a None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:39:04.504622 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9724ab-50c0-4a72-a751-065b2d71926a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:39:04.538879 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-9e9724ab-50c0-4a72-a751-065b2d71926a None None] MFA Rules not processed for user `a0e4900b6ad24eb19fe169d9c8d8c3a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:39:04.540048 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-9e9724ab-50c0-4a72-a751-065b2d71926a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:39:04.663738 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1180/2328] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:39:04 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:39:04.665119 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5c4353-7a5e-470a-83c9-f3bf8fb80082 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:39:04.680077 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5c4353-7a5e-470a-83c9-f3bf8fb80082 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:39:04.685125 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564bf02d-b229-4c21-aaee-991f6740cca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:39:04.685931 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564bf02d-b229-4c21-aaee-991f6740cca4 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:39:04.686225 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564bf02d-b229-4c21-aaee-991f6740cca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:39:04.686509 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564bf02d-b229-4c21-aaee-991f6740cca4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:39:04.687216 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564bf02d-b229-4c21-aaee-991f6740cca4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:39:04.690469 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564bf02d-b229-4c21-aaee-991f6740cca4 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:39:04.746500 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:39:04.770453 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1149/2329] 10.222.0.99 () {60 vars in 1152 bytes} [Thu Jul 14 15:39:04 2022] POST /identity/v3/projects => generated 379 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:39:04.771234 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9724ab-50c0-4a72-a751-065b2d71926a None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:39:04.784938 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9724ab-50c0-4a72-a751-065b2d71926a None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:39:04.864286 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66667354-f244-4a06-b8bf-e95c4e812e74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:39:04.864990 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66667354-f244-4a06-b8bf-e95c4e812e74 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:39:04.865282 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66667354-f244-4a06-b8bf-e95c4e812e74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:39:04.865569 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66667354-f244-4a06-b8bf-e95c4e812e74 admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:39:04.866283 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66667354-f244-4a06-b8bf-e95c4e812e74 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:39:04.868281 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66667354-f244-4a06-b8bf-e95c4e812e74 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:39:04.895541 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1181/2330] 10.222.0.99 () {60 vars in 1143 bytes} [Thu Jul 14 15:39:04 2022] POST /identity/v3/users => generated 296 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 15:39:04.899940 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564bf02d-b229-4c21-aaee-991f6740cca4 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:39:04.915121 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564bf02d-b229-4c21-aaee-991f6740cca4 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:39:04.920984 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9928b4-806e-4ef6-b0f0-5fe40eb0cee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:39:04.921685 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9928b4-806e-4ef6-b0f0-5fe40eb0cee5 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:39:04.922006 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9928b4-806e-4ef6-b0f0-5fe40eb0cee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:39:04.922476 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9928b4-806e-4ef6-b0f0-5fe40eb0cee5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:39:04.923713 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9928b4-806e-4ef6-b0f0-5fe40eb0cee5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:39:04.925365 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9928b4-806e-4ef6-b0f0-5fe40eb0cee5 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:39:04.937999 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1150/2331] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:39:04 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 15:39:04.942956 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66667354-f244-4a06-b8bf-e95c4e812e74 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:39:04.955380 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66667354-f244-4a06-b8bf-e95c4e812e74 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:39:04.965175 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d41edbe-bc85-4d23-91a5-3b7eeab19e87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:39:04.966100 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d41edbe-bc85-4d23-91a5-3b7eeab19e87 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:39:04.968009 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d41edbe-bc85-4d23-91a5-3b7eeab19e87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:39:04.968411 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d41edbe-bc85-4d23-91a5-3b7eeab19e87 admin admin] PATH_INFO: `/v3/projects/5f1c8e0732c4436ca32e38bd3b352e2a/users/99ed6f809fbc4b758e26f55c0e637e67/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:39:04.989619 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d41edbe-bc85-4d23-91a5-3b7eeab19e87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f1c8e0732c4436ca32e38bd3b352e2a, user_id=99ed6f809fbc4b758e26f55c0e637e67, role_id=d0f6ba1314f14d9f9c816bdb5d0c3aac)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:39:04.993162 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d41edbe-bc85-4d23-91a5-3b7eeab19e87 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:39:04.995705 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:39:05.011052 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1182/2332] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 15:39:04 2022] PUT /identity/v3/projects/5f1c8e0732c4436ca32e38bd3b352e2a/users/99ed6f809fbc4b758e26f55c0e637e67/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:39:05.015005 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9928b4-806e-4ef6-b0f0-5fe40eb0cee5 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:39:05.025959 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9928b4-806e-4ef6-b0f0-5fe40eb0cee5 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:39:05.100753 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d16c63f-7cd0-470a-b826-aafd7168306e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:39:05.101542 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d16c63f-7cd0-470a-b826-aafd7168306e admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:39:05.101910 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d16c63f-7cd0-470a-b826-aafd7168306e admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:39:05.102256 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d16c63f-7cd0-470a-b826-aafd7168306e admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:39:05.103006 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d16c63f-7cd0-470a-b826-aafd7168306e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:39:05.104693 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d16c63f-7cd0-470a-b826-aafd7168306e admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:39:05.124701 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1151/2333] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:39:05 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 15:39:05.125943 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d41edbe-bc85-4d23-91a5-3b7eeab19e87 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:39:05.136624 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d41edbe-bc85-4d23-91a5-3b7eeab19e87 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:39:05.142430 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b67a71a-a95e-48fa-b9ba-4dcd302e0bba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:39:05.143475 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b67a71a-a95e-48fa-b9ba-4dcd302e0bba admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:39:05.143812 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b67a71a-a95e-48fa-b9ba-4dcd302e0bba admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:39:05.144097 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b67a71a-a95e-48fa-b9ba-4dcd302e0bba admin admin] PATH_INFO: `/v3/projects/5f1c8e0732c4436ca32e38bd3b352e2a/users/99ed6f809fbc4b758e26f55c0e637e67/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:39:05.148577 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b67a71a-a95e-48fa-b9ba-4dcd302e0bba admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f1c8e0732c4436ca32e38bd3b352e2a, user_id=99ed6f809fbc4b758e26f55c0e637e67, role_id=d0f6ba1314f14d9f9c816bdb5d0c3aac)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:39:05.152584 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b67a71a-a95e-48fa-b9ba-4dcd302e0bba admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:39:05.190836 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1183/2334] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 15:39:05 2022] PUT /identity/v3/projects/5f1c8e0732c4436ca32e38bd3b352e2a/users/99ed6f809fbc4b758e26f55c0e637e67/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:39:11.149775 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c704d4-b14e-44af-80fc-6f7210f3b9f4 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:39:11.150706 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c704d4-b14e-44af-80fc-6f7210f3b9f4 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:39:11.151146 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c704d4-b14e-44af-80fc-6f7210f3b9f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:39:11.200456 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-77c704d4-b14e-44af-80fc-6f7210f3b9f4 None None] MFA Rules not processed for user `99ed6f809fbc4b758e26f55c0e637e67`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:39:11.201796 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-77c704d4-b14e-44af-80fc-6f7210f3b9f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:39:11.289773 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1152/2335] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:39:11 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:39:11.304067 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b67a71a-a95e-48fa-b9ba-4dcd302e0bba admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:39:11.305930 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-8b67a71a-a95e-48fa-b9ba-4dcd302e0bba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:39:11.387899 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b67a71a-a95e-48fa-b9ba-4dcd302e0bba admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:39:11.394464 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c7cddd-a1dd-40b7-8993-4219ab19b9dc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9b1c2622934c5185f3b5f1ea623bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:39:11.395267 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c7cddd-a1dd-40b7-8993-4219ab19b9dc service nova] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:39:11.395690 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c7cddd-a1dd-40b7-8993-4219ab19b9dc service nova] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:39:11.396004 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c7cddd-a1dd-40b7-8993-4219ab19b9dc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:39:11.462504 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c7cddd-a1dd-40b7-8993-4219ab19b9dc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:39:11.464155 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c7cddd-a1dd-40b7-8993-4219ab19b9dc service nova] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:39:11.474797 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1184/2336] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:39:11 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:39:18.993265 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ae5a5e-1287-4349-9fca-995633944a87 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:39:18.993797 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ae5a5e-1287-4349-9fca-995633944a87 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:39:18.993797 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ae5a5e-1287-4349-9fca-995633944a87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:39:19.013343 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-44ae5a5e-1287-4349-9fca-995633944a87 None None] MFA Rules not processed for user `1f3411c8a7454203a4fbd748f1f3031a`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:39:19.014568 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-44ae5a5e-1287-4349-9fca-995633944a87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:39:19.071770 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1153/2337] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:39:18 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:39:21.469843 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:INSERT=2,keystone:SELECT=63 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:39:29.065080 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=93 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:39:29.980598 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c7cddd-a1dd-40b7-8993-4219ab19b9dc service nova] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:39:29.982210 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-45c7cddd-a1dd-40b7-8993-4219ab19b9dc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:39:30.076322 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c7cddd-a1dd-40b7-8993-4219ab19b9dc service nova] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:39:30.081773 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372d4123-8542-4b09-944f-6960bd695cab service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc157c485f794e968a60d8405c9310ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:39:30.082441 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372d4123-8542-4b09-944f-6960bd695cab service placement] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:39:30.082733 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372d4123-8542-4b09-944f-6960bd695cab service placement] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:39:30.083019 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372d4123-8542-4b09-944f-6960bd695cab service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:39:30.084927 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-372d4123-8542-4b09-944f-6960bd695cab service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:39:30.097672 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372d4123-8542-4b09-944f-6960bd695cab service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:39:30.099166 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372d4123-8542-4b09-944f-6960bd695cab service placement] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:39:30.110446 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1185/2338] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:39:29 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:39:40.105461 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=32 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:39:41.496100 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d13a715-8135-4e90-bc19-bbd69a40760e None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:39:41.496669 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d13a715-8135-4e90-bc19-bbd69a40760e None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:39:41.496669 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d13a715-8135-4e90-bc19-bbd69a40760e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:39:41.516153 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-6d13a715-8135-4e90-bc19-bbd69a40760e None None] MFA Rules not processed for user `d01c301ea20d4acab8dff1148501a975`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:39:41.517972 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-6d13a715-8135-4e90-bc19-bbd69a40760e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:39:41.576200 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1154/2339] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:39:41 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:39:47.753155 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372d4123-8542-4b09-944f-6960bd695cab service placement] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:39:47.756041 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-372d4123-8542-4b09-944f-6960bd695cab service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:39:47.833926 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372d4123-8542-4b09-944f-6960bd695cab service placement] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:39:47.838523 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1580e4e9-1531-44ee-a432-60882c049c88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:39:47.839420 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1580e4e9-1531-44ee-a432-60882c049c88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:39:47.839742 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1580e4e9-1531-44ee-a432-60882c049c88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:39:47.840024 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1580e4e9-1531-44ee-a432-60882c049c88 admin admin] PATH_INFO: `/v3/users/26f55cdaab19406b82d0ce4b4886980d` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:39:47.841267 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1580e4e9-1531-44ee-a432-60882c049c88 admin admin] RBAC: Authorizing `identity:delete_user(user_id=26f55cdaab19406b82d0ce4b4886980d)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:39:47.843213 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1580e4e9-1531-44ee-a432-60882c049c88 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:39:47.896775 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-1580e4e9-1531-44ee-a432-60882c049c88 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26f55cdaab19406b82d0ce4b4886980d'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:39:47.911881 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-1580e4e9-1531-44ee-a432-60882c049c88 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26f55cdaab19406b82d0ce4b4886980d'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:39:47.928961 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-1580e4e9-1531-44ee-a432-60882c049c88 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26f55cdaab19406b82d0ce4b4886980d'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:39:47.930437 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-1580e4e9-1531-44ee-a432-60882c049c88 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26f55cdaab19406b82d0ce4b4886980d'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:39:47.931285 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-1580e4e9-1531-44ee-a432-60882c049c88 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26f55cdaab19406b82d0ce4b4886980d'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:39:47.946309 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1186/2340] 10.222.0.99 () {58 vars in 1223 bytes} [Thu Jul 14 15:39:47 2022] DELETE /identity/v3/users/26f55cdaab19406b82d0ce4b4886980d => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:39:47.950880 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c166e994-7189-468d-bdc5-0ad93083c856 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:39:47.951405 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c166e994-7189-468d-bdc5-0ad93083c856 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:39:47.951491 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c166e994-7189-468d-bdc5-0ad93083c856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:39:47.971961 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-c166e994-7189-468d-bdc5-0ad93083c856 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:39:47.973206 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c166e994-7189-468d-bdc5-0ad93083c856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:39:48.045306 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1155/2341] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:39:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:39:48.055812 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1580e4e9-1531-44ee-a432-60882c049c88 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:39:48.057263 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-1580e4e9-1531-44ee-a432-60882c049c88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:39:48.162262 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1580e4e9-1531-44ee-a432-60882c049c88 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:39:48.167441 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e46cb6d-86f6-4de3-a342-1ddcf5aea811 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:39:48.168349 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e46cb6d-86f6-4de3-a342-1ddcf5aea811 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:39:48.168832 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e46cb6d-86f6-4de3-a342-1ddcf5aea811 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:39:48.169115 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e46cb6d-86f6-4de3-a342-1ddcf5aea811 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:39:48.181737 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e46cb6d-86f6-4de3-a342-1ddcf5aea811 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:39:48.183258 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e46cb6d-86f6-4de3-a342-1ddcf5aea811 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:39:48.195254 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1187/2342] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:39:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:39:48.586935 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c166e994-7189-468d-bdc5-0ad93083c856 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:39:48.589166 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c166e994-7189-468d-bdc5-0ad93083c856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:39:48.604119 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c166e994-7189-468d-bdc5-0ad93083c856 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:39:48.610823 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d434ef-9f72-4f52-b8bd-292d5abe43bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:39:48.612288 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d434ef-9f72-4f52-b8bd-292d5abe43bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:39:48.612573 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d434ef-9f72-4f52-b8bd-292d5abe43bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:39:48.612848 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d434ef-9f72-4f52-b8bd-292d5abe43bc admin admin] PATH_INFO: `/v3/projects/455c85fb9b5e45218ae7076a682ecaba` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:39:48.614796 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d434ef-9f72-4f52-b8bd-292d5abe43bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=455c85fb9b5e45218ae7076a682ecaba)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:39:48.616789 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d434ef-9f72-4f52-b8bd-292d5abe43bc admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:39:48.666781 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d4d434ef-9f72-4f52-b8bd-292d5abe43bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '455c85fb9b5e45218ae7076a682ecaba', 'request_id': 'req-d4d434ef-9f72-4f52-b8bd-292d5abe43bc'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:39:48.673738 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d4d434ef-9f72-4f52-b8bd-292d5abe43bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '455c85fb9b5e45218ae7076a682ecaba', 'request_id': 'req-d4d434ef-9f72-4f52-b8bd-292d5abe43bc'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:39:48.679531 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d4d434ef-9f72-4f52-b8bd-292d5abe43bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '455c85fb9b5e45218ae7076a682ecaba', 'request_id': 'req-d4d434ef-9f72-4f52-b8bd-292d5abe43bc'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:39:48.693147 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d4d434ef-9f72-4f52-b8bd-292d5abe43bc admin admin] The token cache is being invalidate because project 455c85fb9b5e45218ae7076a682ecaba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 15:39:48.693431 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d4d434ef-9f72-4f52-b8bd-292d5abe43bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:39:48.694079 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d4d434ef-9f72-4f52-b8bd-292d5abe43bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:39:48.695645 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1156/2343] 10.222.0.99 () {58 vars in 1232 bytes} [Thu Jul 14 15:39:48 2022] DELETE /identity/v3/projects/455c85fb9b5e45218ae7076a682ecaba => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:39:48.700577 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8068aeaa-a93d-48a6-85b1-70111f74d579 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:39:48.700896 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8068aeaa-a93d-48a6-85b1-70111f74d579 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:39:48.701177 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8068aeaa-a93d-48a6-85b1-70111f74d579 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:39:48.719811 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-8068aeaa-a93d-48a6-85b1-70111f74d579 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:39:48.721254 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-8068aeaa-a93d-48a6-85b1-70111f74d579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:39:48.745180 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:39:48.796150 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1188/2344] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:39:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:39:48.805284 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506a40a9-02a3-4ba5-a70f-8d707393d486 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:39:48.805582 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506a40a9-02a3-4ba5-a70f-8d707393d486 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:39:48.805869 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506a40a9-02a3-4ba5-a70f-8d707393d486 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:39:48.825850 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-506a40a9-02a3-4ba5-a70f-8d707393d486 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:39:48.827184 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-506a40a9-02a3-4ba5-a70f-8d707393d486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:39:48.836042 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1157/2345] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:39:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:39:48.840513 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8068aeaa-a93d-48a6-85b1-70111f74d579 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:39:48.851876 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8068aeaa-a93d-48a6-85b1-70111f74d579 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:39:48.856744 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d02089-b109-4c29-b041-743b82538559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:39:48.857453 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d02089-b109-4c29-b041-743b82538559 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:39:48.857744 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d02089-b109-4c29-b041-743b82538559 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:39:48.858026 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d02089-b109-4c29-b041-743b82538559 admin admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:39:48.858774 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d02089-b109-4c29-b041-743b82538559 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:39:48.859086 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d02089-b109-4c29-b041-743b82538559 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:39:48.861726 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d02089-b109-4c29-b041-743b82538559 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:39:48.881506 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1189/2346] 10.222.0.99 () {58 vars in 1152 bytes} [Thu Jul 14 15:39:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:39:48.886001 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506a40a9-02a3-4ba5-a70f-8d707393d486 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:39:48.895959 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506a40a9-02a3-4ba5-a70f-8d707393d486 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:39:48.900994 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1825a7-f2ae-4fc6-9831-a113edea28ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:39:48.901658 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1825a7-f2ae-4fc6-9831-a113edea28ea admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:39:48.901945 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1825a7-f2ae-4fc6-9831-a113edea28ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:39:48.902260 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1825a7-f2ae-4fc6-9831-a113edea28ea admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:39:48.902956 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1825a7-f2ae-4fc6-9831-a113edea28ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:39:48.904907 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1825a7-f2ae-4fc6-9831-a113edea28ea admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:39:48.927266 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:39:48.940907 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1158/2347] 10.222.0.99 () {60 vars in 1152 bytes} [Thu Jul 14 15:39:48 2022] POST /identity/v3/projects => generated 375 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 15:39:48.942657 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d02089-b109-4c29-b041-743b82538559 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:39:48.958857 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d02089-b109-4c29-b041-743b82538559 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:39:49.028506 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00726a1-1087-4db5-aaa6-b0c4bf55eb23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:39:49.029387 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00726a1-1087-4db5-aaa6-b0c4bf55eb23 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:39:49.029498 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00726a1-1087-4db5-aaa6-b0c4bf55eb23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:39:49.029766 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00726a1-1087-4db5-aaa6-b0c4bf55eb23 admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:39:49.030476 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00726a1-1087-4db5-aaa6-b0c4bf55eb23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:39:49.032998 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00726a1-1087-4db5-aaa6-b0c4bf55eb23 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:39:49.056724 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1190/2348] 10.222.0.99 () {60 vars in 1143 bytes} [Thu Jul 14 15:39:48 2022] POST /identity/v3/users => generated 288 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:39:49.061114 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1825a7-f2ae-4fc6-9831-a113edea28ea admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:39:49.072064 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1825a7-f2ae-4fc6-9831-a113edea28ea admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:39:49.076895 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8707747-fe1b-49fe-89cf-be447098d450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:39:49.077568 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8707747-fe1b-49fe-89cf-be447098d450 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:39:49.077857 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8707747-fe1b-49fe-89cf-be447098d450 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:39:49.078140 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8707747-fe1b-49fe-89cf-be447098d450 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:39:49.078870 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8707747-fe1b-49fe-89cf-be447098d450 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:39:49.081268 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8707747-fe1b-49fe-89cf-be447098d450 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:39:49.092445 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1159/2349] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:39:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:39:49.097621 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00726a1-1087-4db5-aaa6-b0c4bf55eb23 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:39:49.108593 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00726a1-1087-4db5-aaa6-b0c4bf55eb23 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:39:49.114643 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba3b12b-9464-4213-a3fe-650755812677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:39:49.115331 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba3b12b-9464-4213-a3fe-650755812677 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:39:49.115649 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba3b12b-9464-4213-a3fe-650755812677 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:39:49.115944 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba3b12b-9464-4213-a3fe-650755812677 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:39:49.116679 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba3b12b-9464-4213-a3fe-650755812677 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:39:49.118385 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba3b12b-9464-4213-a3fe-650755812677 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:39:49.129818 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1191/2350] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:39:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:39:49.134057 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8707747-fe1b-49fe-89cf-be447098d450 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:39:49.145921 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8707747-fe1b-49fe-89cf-be447098d450 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:39:49.150946 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d143b687-39fb-4f1b-b24b-bf771d20a888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:39:49.151874 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d143b687-39fb-4f1b-b24b-bf771d20a888 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:39:49.152161 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d143b687-39fb-4f1b-b24b-bf771d20a888 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:39:49.152444 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d143b687-39fb-4f1b-b24b-bf771d20a888 admin admin] PATH_INFO: `/v3/projects/75292a36c8354a2dbd28428efb2e9df9/users/4689786287394deb9ecafb9f2cdbca89/roles/b8c98fddc0fd4436b577fd1d3361bc38` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:39:49.171199 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d143b687-39fb-4f1b-b24b-bf771d20a888 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75292a36c8354a2dbd28428efb2e9df9, user_id=4689786287394deb9ecafb9f2cdbca89, role_id=b8c98fddc0fd4436b577fd1d3361bc38)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:39:49.174724 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d143b687-39fb-4f1b-b24b-bf771d20a888 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:39:49.194303 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1160/2351] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 15:39:49 2022] PUT /identity/v3/projects/75292a36c8354a2dbd28428efb2e9df9/users/4689786287394deb9ecafb9f2cdbca89/roles/b8c98fddc0fd4436b577fd1d3361bc38 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:39:49.271747 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10afb05-b911-4d46-a20b-9a6967483b79 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:39:49.271747 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10afb05-b911-4d46-a20b-9a6967483b79 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:39:49.271747 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10afb05-b911-4d46-a20b-9a6967483b79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:39:49.298986 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-e10afb05-b911-4d46-a20b-9a6967483b79 None None] MFA Rules not processed for user `4689786287394deb9ecafb9f2cdbca89`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:39:49.300427 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-e10afb05-b911-4d46-a20b-9a6967483b79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:39:49.375119 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1192/2352] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:39:49 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:39:49.387312 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d143b687-39fb-4f1b-b24b-bf771d20a888 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:39:49.391310 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d143b687-39fb-4f1b-b24b-bf771d20a888 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:39:49.456520 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d143b687-39fb-4f1b-b24b-bf771d20a888 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:39:49.460224 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0990e8-1a6e-45c2-adff-90995049829a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3e1a2b83f3d4d88ba5084dfaffd22f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:39:49.460911 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0990e8-1a6e-45c2-adff-90995049829a service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:39:49.461203 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0990e8-1a6e-45c2-adff-90995049829a service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:39:49.463472 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0990e8-1a6e-45c2-adff-90995049829a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:39:49.473028 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0990e8-1a6e-45c2-adff-90995049829a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:39:49.474476 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0990e8-1a6e-45c2-adff-90995049829a service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:39:49.484013 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1161/2353] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:39:49 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:39:50.869589 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2556d1b6-a57f-4971-a48d-babeb66562ad None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:39:50.870133 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2556d1b6-a57f-4971-a48d-babeb66562ad None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:39:50.870133 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2556d1b6-a57f-4971-a48d-babeb66562ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:39:50.888663 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-2556d1b6-a57f-4971-a48d-babeb66562ad None None] MFA Rules not processed for user `4689786287394deb9ecafb9f2cdbca89`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:39:50.889878 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-2556d1b6-a57f-4971-a48d-babeb66562ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:39:50.896910 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1193/2354] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:39:50 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:39:59.479572 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=41,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:40:00.883100 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=92,keystone:INSERT=3 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:40:08.956284 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d62332-47f9-46aa-8656-21294ade69a8 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:40:08.956812 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d62332-47f9-46aa-8656-21294ade69a8 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:40:08.956918 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d62332-47f9-46aa-8656-21294ade69a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:40:08.977967 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-42d62332-47f9-46aa-8656-21294ade69a8 None None] MFA Rules not processed for user `a0e4900b6ad24eb19fe169d9c8d8c3a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:40:08.979184 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-42d62332-47f9-46aa-8656-21294ade69a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:40:09.038156 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1162/2355] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:40:08 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:40:10.261350 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a5eaf9-fd36-4757-829a-09d9c01ac452 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:40:10.261832 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a5eaf9-fd36-4757-829a-09d9c01ac452 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:40:10.261931 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a5eaf9-fd36-4757-829a-09d9c01ac452 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:40:10.280099 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-c6a5eaf9-fd36-4757-829a-09d9c01ac452 None None] MFA Rules not processed for user `adaeadf613954d7393b3fd3bae68c362`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:40:10.281336 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-c6a5eaf9-fd36-4757-829a-09d9c01ac452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:40:10.341906 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1194/2356] 10.222.0.99 () {58 vars in 1009 bytes} [Thu Jul 14 15:40:10 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:40:10.385584 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d62332-47f9-46aa-8656-21294ade69a8 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:40:10.387135 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-42d62332-47f9-46aa-8656-21294ade69a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:40:10.487972 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d62332-47f9-46aa-8656-21294ade69a8 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:40:10.494480 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a08fbd-e775-4bc5-87a6-d64b1602970c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd2ec3b3ebc4aacb322c95270374451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:40:10.495233 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a08fbd-e775-4bc5-87a6-d64b1602970c service swift] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:40:10.495587 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a08fbd-e775-4bc5-87a6-d64b1602970c service swift] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:40:10.496019 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a08fbd-e775-4bc5-87a6-d64b1602970c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:40:10.507288 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a08fbd-e775-4bc5-87a6-d64b1602970c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:40:10.509906 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a08fbd-e775-4bc5-87a6-d64b1602970c service swift] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:40:10.521108 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1163/2357] 10.222.0.99 () {60 vars in 1384 bytes} [Thu Jul 14 15:40:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 15:40:13.485754 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039f277e-276c-4ced-9b5e-ebd4724e5d40 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:40:13.486439 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039f277e-276c-4ced-9b5e-ebd4724e5d40 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:40:13.486547 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039f277e-276c-4ced-9b5e-ebd4724e5d40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:40:13.557533 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-039f277e-276c-4ced-9b5e-ebd4724e5d40 None None] MFA Rules not processed for user `adaeadf613954d7393b3fd3bae68c362`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:40:13.570443 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-039f277e-276c-4ced-9b5e-ebd4724e5d40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:40:13.590927 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1195/2358] 10.222.0.99 () {58 vars in 1009 bytes} [Thu Jul 14 15:40:13 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:40:13.606944 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a08fbd-e775-4bc5-87a6-d64b1602970c service swift] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:40:13.632956 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a08fbd-e775-4bc5-87a6-d64b1602970c service swift] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:40:13.662555 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9e8fd9-0a2c-4259-9b6e-f5cb0c06be5e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd2ec3b3ebc4aacb322c95270374451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:40:13.667349 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9e8fd9-0a2c-4259-9b6e-f5cb0c06be5e service swift] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:40:13.670158 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9e8fd9-0a2c-4259-9b6e-f5cb0c06be5e service swift] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:40:13.671079 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9e8fd9-0a2c-4259-9b6e-f5cb0c06be5e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:40:13.705541 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9e8fd9-0a2c-4259-9b6e-f5cb0c06be5e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:40:13.708748 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9e8fd9-0a2c-4259-9b6e-f5cb0c06be5e service swift] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:40:13.723778 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1164/2359] 10.222.0.99 () {60 vars in 1384 bytes} [Thu Jul 14 15:40:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 14 15:40:23.513197 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=28 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:40:23.718283 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=63 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:40:35.210132 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcac3d4e-4297-422a-8d81-be99e9a22bc2 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:40:35.210678 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcac3d4e-4297-422a-8d81-be99e9a22bc2 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:40:35.210678 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcac3d4e-4297-422a-8d81-be99e9a22bc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:40:35.234640 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-fcac3d4e-4297-422a-8d81-be99e9a22bc2 None None] MFA Rules not processed for user `adaeadf613954d7393b3fd3bae68c362`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:40:35.236064 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-fcac3d4e-4297-422a-8d81-be99e9a22bc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:40:35.245573 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1196/2360] 10.222.0.99 () {58 vars in 967 bytes} [Thu Jul 14 15:40:35 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 15:40:35.258808 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9e8fd9-0a2c-4259-9b6e-f5cb0c06be5e service swift] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:40:35.270693 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9e8fd9-0a2c-4259-9b6e-f5cb0c06be5e service swift] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:40:35.276245 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e8b70b-bbf3-4717-a536-eb0606498e86 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd2ec3b3ebc4aacb322c95270374451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:40:35.276947 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e8b70b-bbf3-4717-a536-eb0606498e86 service swift] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:40:35.277247 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e8b70b-bbf3-4717-a536-eb0606498e86 service swift] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:40:35.277534 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e8b70b-bbf3-4717-a536-eb0606498e86 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:40:35.288245 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e8b70b-bbf3-4717-a536-eb0606498e86 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:40:35.289957 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e8b70b-bbf3-4717-a536-eb0606498e86 service swift] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:40:35.299786 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1165/2361] 10.222.0.99 () {60 vars in 1384 bytes} [Thu Jul 14 15:40:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 15:40:37.721473 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc1936f-6b4d-4e6c-8932-71ea8ccccb24 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:40:37.721982 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc1936f-6b4d-4e6c-8932-71ea8ccccb24 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:40:37.722093 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc1936f-6b4d-4e6c-8932-71ea8ccccb24 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:40:37.724155 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1197/2362] 10.222.0.99 () {54 vars in 921 bytes} [Thu Jul 14 15:40:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:40:37.734745 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3659601c-8d1b-4205-98d1-3231a07f02f8 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:40:37.735237 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3659601c-8d1b-4205-98d1-3231a07f02f8 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:40:37.736445 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3659601c-8d1b-4205-98d1-3231a07f02f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:40:37.755549 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-3659601c-8d1b-4205-98d1-3231a07f02f8 None None] MFA Rules not processed for user `dc157c485f794e968a60d8405c9310ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:40:37.756819 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-3659601c-8d1b-4205-98d1-3231a07f02f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:40:37.816434 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc1936f-6b4d-4e6c-8932-71ea8ccccb24 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:40:37.818363 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-5fc1936f-6b4d-4e6c-8932-71ea8ccccb24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:40:37.834209 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1166/2363] 10.222.0.99 () {58 vars in 1018 bytes} [Thu Jul 14 15:40:37 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 15:40:37.846746 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3659601c-8d1b-4205-98d1-3231a07f02f8 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:40:37.848202 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-3659601c-8d1b-4205-98d1-3231a07f02f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:40:37.862020 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3659601c-8d1b-4205-98d1-3231a07f02f8 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:40:37.867136 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2e0dd3-a27d-4536-b767-ea3b3fd1eb27 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc157c485f794e968a60d8405c9310ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:40:37.867835 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2e0dd3-a27d-4536-b767-ea3b3fd1eb27 service placement] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:40:37.868126 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2e0dd3-a27d-4536-b767-ea3b3fd1eb27 service placement] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:40:37.868408 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2e0dd3-a27d-4536-b767-ea3b3fd1eb27 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:40:37.878148 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2e0dd3-a27d-4536-b767-ea3b3fd1eb27 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:40:37.879778 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2e0dd3-a27d-4536-b767-ea3b3fd1eb27 service placement] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:40:37.889659 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1167/2364] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:40:37 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 15:40:37.895424 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc1936f-6b4d-4e6c-8932-71ea8ccccb24 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:40:37.900243 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1927c9-425b-4ae4-8bc1-7ba528ae3abf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:40:37.901128 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1927c9-425b-4ae4-8bc1-7ba528ae3abf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:40:37.901415 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1927c9-425b-4ae4-8bc1-7ba528ae3abf admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:40:37.901697 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1927c9-425b-4ae4-8bc1-7ba528ae3abf admin admin] PATH_INFO: `/v3/users/1f3411c8a7454203a4fbd748f1f3031a` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:40:37.902934 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1927c9-425b-4ae4-8bc1-7ba528ae3abf admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f3411c8a7454203a4fbd748f1f3031a)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:40:37.904882 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1927c9-425b-4ae4-8bc1-7ba528ae3abf admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:40:37.952724 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-5e1927c9-425b-4ae4-8bc1-7ba528ae3abf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f3411c8a7454203a4fbd748f1f3031a'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:40:37.962425 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-5e1927c9-425b-4ae4-8bc1-7ba528ae3abf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f3411c8a7454203a4fbd748f1f3031a'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:40:37.974375 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-5e1927c9-425b-4ae4-8bc1-7ba528ae3abf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f3411c8a7454203a4fbd748f1f3031a'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:40:37.975036 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-5e1927c9-425b-4ae4-8bc1-7ba528ae3abf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f3411c8a7454203a4fbd748f1f3031a'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:40:37.975701 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-5e1927c9-425b-4ae4-8bc1-7ba528ae3abf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f3411c8a7454203a4fbd748f1f3031a'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:40:37.988153 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1198/2365] 10.222.0.99 () {58 vars in 1223 bytes} [Thu Jul 14 15:40:37 2022] DELETE /identity/v3/users/1f3411c8a7454203a4fbd748f1f3031a => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:40:38.167191 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2e0dd3-a27d-4536-b767-ea3b3fd1eb27 service placement] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:40:38.179840 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2e0dd3-a27d-4536-b767-ea3b3fd1eb27 service placement] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:40:38.239726 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8bdb5f-6f7b-4803-9035-1843fb0d5ce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:40:38.240577 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8bdb5f-6f7b-4803-9035-1843fb0d5ce1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:40:38.240868 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8bdb5f-6f7b-4803-9035-1843fb0d5ce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:40:38.241151 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8bdb5f-6f7b-4803-9035-1843fb0d5ce1 admin admin] PATH_INFO: `/v3/projects/2345d49f9fee4842826a396c43132a33` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:40:38.242344 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8bdb5f-6f7b-4803-9035-1843fb0d5ce1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2345d49f9fee4842826a396c43132a33)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:40:38.244285 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8bdb5f-6f7b-4803-9035-1843fb0d5ce1 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:40:38.308206 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5b8bdb5f-6f7b-4803-9035-1843fb0d5ce1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2345d49f9fee4842826a396c43132a33', 'request_id': 'req-5b8bdb5f-6f7b-4803-9035-1843fb0d5ce1'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:40:38.327953 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5b8bdb5f-6f7b-4803-9035-1843fb0d5ce1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2345d49f9fee4842826a396c43132a33', 'request_id': 'req-5b8bdb5f-6f7b-4803-9035-1843fb0d5ce1'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:40:38.344949 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5b8bdb5f-6f7b-4803-9035-1843fb0d5ce1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2345d49f9fee4842826a396c43132a33', 'request_id': 'req-5b8bdb5f-6f7b-4803-9035-1843fb0d5ce1'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:40:38.358959 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:40:38.369029 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5b8bdb5f-6f7b-4803-9035-1843fb0d5ce1 admin admin] The token cache is being invalidate because project 2345d49f9fee4842826a396c43132a33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 15:40:38.369249 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5b8bdb5f-6f7b-4803-9035-1843fb0d5ce1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:40:38.369976 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5b8bdb5f-6f7b-4803-9035-1843fb0d5ce1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:40:38.378387 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1168/2366] 10.222.0.99 () {58 vars in 1232 bytes} [Thu Jul 14 15:40:38 2022] DELETE /identity/v3/projects/2345d49f9fee4842826a396c43132a33 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:40:38.378934 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fc5bc8-bdcc-4726-8cca-21d3a601ab9e None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:40:38.379231 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fc5bc8-bdcc-4726-8cca-21d3a601ab9e None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:40:38.379528 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fc5bc8-bdcc-4726-8cca-21d3a601ab9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:40:38.398715 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-70fc5bc8-bdcc-4726-8cca-21d3a601ab9e None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:40:38.399973 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-70fc5bc8-bdcc-4726-8cca-21d3a601ab9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:40:38.463439 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1199/2367] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:40:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:40:38.471885 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ad5430-862b-4f41-a7c1-32b1ba803727 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:40:38.472180 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ad5430-862b-4f41-a7c1-32b1ba803727 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:40:38.472462 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ad5430-862b-4f41-a7c1-32b1ba803727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:40:38.491957 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-a8ad5430-862b-4f41-a7c1-32b1ba803727 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:40:38.493193 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a8ad5430-862b-4f41-a7c1-32b1ba803727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:40:38.501527 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1169/2368] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:40:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 15:40:38.505815 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fc5bc8-bdcc-4726-8cca-21d3a601ab9e None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:40:38.515987 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fc5bc8-bdcc-4726-8cca-21d3a601ab9e None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:40:38.520674 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323a0a11-c3b0-44ad-872a-5bd0636d82e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:40:38.521354 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323a0a11-c3b0-44ad-872a-5bd0636d82e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:40:38.521641 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323a0a11-c3b0-44ad-872a-5bd0636d82e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:40:38.521921 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323a0a11-c3b0-44ad-872a-5bd0636d82e8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:40:38.522658 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323a0a11-c3b0-44ad-872a-5bd0636d82e8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:40:38.522969 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323a0a11-c3b0-44ad-872a-5bd0636d82e8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:40:38.524946 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323a0a11-c3b0-44ad-872a-5bd0636d82e8 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:40:38.534502 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:40:38.548493 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1200/2369] 10.222.0.99 () {58 vars in 1152 bytes} [Thu Jul 14 15:40:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:40:38.548687 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ad5430-862b-4f41-a7c1-32b1ba803727 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:40:38.580616 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ad5430-862b-4f41-a7c1-32b1ba803727 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:40:38.604621 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f6ebd3-9e55-4b63-acb5-717da28c0bbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:40:38.605878 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f6ebd3-9e55-4b63-acb5-717da28c0bbc admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:40:38.606209 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f6ebd3-9e55-4b63-acb5-717da28c0bbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:40:38.606486 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f6ebd3-9e55-4b63-acb5-717da28c0bbc admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:40:38.607272 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f6ebd3-9e55-4b63-acb5-717da28c0bbc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:40:38.610078 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f6ebd3-9e55-4b63-acb5-717da28c0bbc admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:40:38.657921 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1170/2370] 10.222.0.99 () {60 vars in 1152 bytes} [Thu Jul 14 15:40:38 2022] POST /identity/v3/projects => generated 373 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 15:40:38.662099 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323a0a11-c3b0-44ad-872a-5bd0636d82e8 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:40:38.676397 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323a0a11-c3b0-44ad-872a-5bd0636d82e8 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:40:38.756378 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48dbff7-b382-4a40-a4cb-f2f507ebab7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:40:38.757357 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48dbff7-b382-4a40-a4cb-f2f507ebab7d admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:40:38.757651 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48dbff7-b382-4a40-a4cb-f2f507ebab7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:40:38.757991 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48dbff7-b382-4a40-a4cb-f2f507ebab7d admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:40:38.758732 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48dbff7-b382-4a40-a4cb-f2f507ebab7d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:40:38.761334 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48dbff7-b382-4a40-a4cb-f2f507ebab7d admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:40:38.786700 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1201/2371] 10.222.0.99 () {60 vars in 1143 bytes} [Thu Jul 14 15:40:38 2022] POST /identity/v3/users => generated 287 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:40:38.793860 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f6ebd3-9e55-4b63-acb5-717da28c0bbc admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:40:38.811487 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f6ebd3-9e55-4b63-acb5-717da28c0bbc admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:40:38.817359 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4dbeab-5712-4638-bed9-4eaeed90146d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:40:38.818197 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4dbeab-5712-4638-bed9-4eaeed90146d admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:40:38.818617 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4dbeab-5712-4638-bed9-4eaeed90146d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:40:38.818996 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4dbeab-5712-4638-bed9-4eaeed90146d admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:40:38.819837 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4dbeab-5712-4638-bed9-4eaeed90146d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:40:38.821515 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4dbeab-5712-4638-bed9-4eaeed90146d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:40:38.836128 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1171/2372] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:40:38 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:40:38.840058 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48dbff7-b382-4a40-a4cb-f2f507ebab7d admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:40:38.852438 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48dbff7-b382-4a40-a4cb-f2f507ebab7d admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:40:38.857870 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a076c1-8ba5-4388-bc22-853060b12c5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:40:38.858555 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a076c1-8ba5-4388-bc22-853060b12c5f admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:40:38.858852 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a076c1-8ba5-4388-bc22-853060b12c5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:40:38.859140 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a076c1-8ba5-4388-bc22-853060b12c5f admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:40:38.859921 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a076c1-8ba5-4388-bc22-853060b12c5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:40:38.862861 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a076c1-8ba5-4388-bc22-853060b12c5f admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:40:38.880009 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4dbeab-5712-4638-bed9-4eaeed90146d admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:40:38.892059 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4dbeab-5712-4638-bed9-4eaeed90146d admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:40:38.892295 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1202/2373] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:40:38 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:40:38.896200 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36241ce-3018-4ff9-87db-ea3e11bfbe4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:40:38.897103 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36241ce-3018-4ff9-87db-ea3e11bfbe4e admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:40:38.897395 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36241ce-3018-4ff9-87db-ea3e11bfbe4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:40:38.897681 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36241ce-3018-4ff9-87db-ea3e11bfbe4e admin admin] PATH_INFO: `/v3/projects/16f5da37d8ec473dafd80531f7f60c89/users/dbeb3476ffee4611896156633c84265c/roles/b8c98fddc0fd4436b577fd1d3361bc38` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:40:38.915269 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36241ce-3018-4ff9-87db-ea3e11bfbe4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=16f5da37d8ec473dafd80531f7f60c89, user_id=dbeb3476ffee4611896156633c84265c, role_id=b8c98fddc0fd4436b577fd1d3361bc38)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:40:38.918049 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36241ce-3018-4ff9-87db-ea3e11bfbe4e admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:40:38.935835 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1172/2374] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 15:40:38 2022] PUT /identity/v3/projects/16f5da37d8ec473dafd80531f7f60c89/users/dbeb3476ffee4611896156633c84265c/roles/b8c98fddc0fd4436b577fd1d3361bc38 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:40:38.941584 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37a3c52-736c-4258-a459-a0574a591570 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:40:38.941881 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37a3c52-736c-4258-a459-a0574a591570 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:40:38.942168 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37a3c52-736c-4258-a459-a0574a591570 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:40:38.961473 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-f37a3c52-736c-4258-a459-a0574a591570 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:40:38.962731 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-f37a3c52-736c-4258-a459-a0574a591570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:40:39.031142 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1203/2375] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:40:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:40:39.040936 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36241ce-3018-4ff9-87db-ea3e11bfbe4e admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:40:39.042647 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c36241ce-3018-4ff9-87db-ea3e11bfbe4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:40:39.099798 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36241ce-3018-4ff9-87db-ea3e11bfbe4e admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:40:39.103320 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf331e37-50db-4789-b25c-c94775ef93e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:40:39.103990 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf331e37-50db-4789-b25c-c94775ef93e9 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:40:39.104277 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf331e37-50db-4789-b25c-c94775ef93e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:40:39.104556 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf331e37-50db-4789-b25c-c94775ef93e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:40:39.113836 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf331e37-50db-4789-b25c-c94775ef93e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:40:39.115253 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf331e37-50db-4789-b25c-c94775ef93e9 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:40:39.124598 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1173/2376] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:40:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:40:44.470741 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19963c6-e551-47ed-95ac-1768896564bf None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:40:44.471231 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19963c6-e551-47ed-95ac-1768896564bf None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:40:44.471332 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19963c6-e551-47ed-95ac-1768896564bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:40:44.503759 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-f19963c6-e551-47ed-95ac-1768896564bf None None] MFA Rules not processed for user `dbeb3476ffee4611896156633c84265c`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:40:44.505404 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-f19963c6-e551-47ed-95ac-1768896564bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:40:44.594430 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1204/2377] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:40:44 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:40:44.595285 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf331e37-50db-4789-b25c-c94775ef93e9 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:40:44.608077 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf331e37-50db-4789-b25c-c94775ef93e9 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:40:44.615920 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e88a1bc-9fc3-4be4-872a-5f4154cf575e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:40:44.616906 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e88a1bc-9fc3-4be4-872a-5f4154cf575e admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:40:44.617288 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e88a1bc-9fc3-4be4-872a-5f4154cf575e admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:40:44.617753 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e88a1bc-9fc3-4be4-872a-5f4154cf575e admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:40:44.618655 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e88a1bc-9fc3-4be4-872a-5f4154cf575e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:40:44.620792 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e88a1bc-9fc3-4be4-872a-5f4154cf575e admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:40:44.662826 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1174/2378] 10.222.0.99 () {60 vars in 1152 bytes} [Thu Jul 14 15:40:44 2022] POST /identity/v3/projects => generated 373 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 15:40:44.666682 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19963c6-e551-47ed-95ac-1768896564bf None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:40:44.679531 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19963c6-e551-47ed-95ac-1768896564bf None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:40:44.685154 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:40:44.767401 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c728049b-08fe-4f34-acc8-df05e9d4a90a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:40:44.768135 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c728049b-08fe-4f34-acc8-df05e9d4a90a admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:40:44.768426 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c728049b-08fe-4f34-acc8-df05e9d4a90a admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:40:44.768703 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c728049b-08fe-4f34-acc8-df05e9d4a90a admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:40:44.769413 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c728049b-08fe-4f34-acc8-df05e9d4a90a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:40:44.771412 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c728049b-08fe-4f34-acc8-df05e9d4a90a admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:40:44.801617 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1205/2379] 10.222.0.99 () {60 vars in 1143 bytes} [Thu Jul 14 15:40:44 2022] POST /identity/v3/users => generated 293 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:40:44.803391 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e88a1bc-9fc3-4be4-872a-5f4154cf575e admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:40:44.818278 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e88a1bc-9fc3-4be4-872a-5f4154cf575e admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:40:44.823390 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb0db5c-1c3b-4ff5-9aa3-ac01fb5e9800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:40:44.824139 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb0db5c-1c3b-4ff5-9aa3-ac01fb5e9800 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:40:44.825131 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb0db5c-1c3b-4ff5-9aa3-ac01fb5e9800 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:40:44.825842 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb0db5c-1c3b-4ff5-9aa3-ac01fb5e9800 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:40:44.826832 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb0db5c-1c3b-4ff5-9aa3-ac01fb5e9800 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:40:44.828504 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb0db5c-1c3b-4ff5-9aa3-ac01fb5e9800 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:40:44.841346 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1175/2380] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:40:44 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:40:44.846214 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c728049b-08fe-4f34-acc8-df05e9d4a90a admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:40:44.856343 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c728049b-08fe-4f34-acc8-df05e9d4a90a admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:40:44.862122 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5dce77-4b82-41d1-8929-9ecf31c595f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:40:44.863105 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5dce77-4b82-41d1-8929-9ecf31c595f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:40:44.863395 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5dce77-4b82-41d1-8929-9ecf31c595f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:40:44.863713 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5dce77-4b82-41d1-8929-9ecf31c595f7 admin admin] PATH_INFO: `/v3/projects/06dedb413f94489b8009e2db06167a88/users/b475d2d866b34290a64141613946dc30/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:40:44.878623 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5dce77-4b82-41d1-8929-9ecf31c595f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06dedb413f94489b8009e2db06167a88, user_id=b475d2d866b34290a64141613946dc30, role_id=d0f6ba1314f14d9f9c816bdb5d0c3aac)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:40:44.881412 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5dce77-4b82-41d1-8929-9ecf31c595f7 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:40:44.894923 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1206/2381] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 15:40:44 2022] PUT /identity/v3/projects/06dedb413f94489b8009e2db06167a88/users/b475d2d866b34290a64141613946dc30/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:40:44.899156 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb0db5c-1c3b-4ff5-9aa3-ac01fb5e9800 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:40:44.910238 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb0db5c-1c3b-4ff5-9aa3-ac01fb5e9800 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:40:44.971124 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b575fad7-0ed5-422c-ae55-0f699e8a52af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:40:44.971852 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b575fad7-0ed5-422c-ae55-0f699e8a52af admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:40:44.972151 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b575fad7-0ed5-422c-ae55-0f699e8a52af admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:40:44.972433 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b575fad7-0ed5-422c-ae55-0f699e8a52af admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:40:44.973180 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b575fad7-0ed5-422c-ae55-0f699e8a52af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:40:44.974800 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b575fad7-0ed5-422c-ae55-0f699e8a52af admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:40:44.986236 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1176/2382] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 15:40:44 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:40:44.990473 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5dce77-4b82-41d1-8929-9ecf31c595f7 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:40:45.002325 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5dce77-4b82-41d1-8929-9ecf31c595f7 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:40:45.007036 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db25fe62-2871-4163-94fd-fc2d139fc759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:40:45.007958 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db25fe62-2871-4163-94fd-fc2d139fc759 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:40:45.008273 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db25fe62-2871-4163-94fd-fc2d139fc759 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:40:45.008552 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db25fe62-2871-4163-94fd-fc2d139fc759 admin admin] PATH_INFO: `/v3/projects/06dedb413f94489b8009e2db06167a88/users/b475d2d866b34290a64141613946dc30/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:40:45.010530 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db25fe62-2871-4163-94fd-fc2d139fc759 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06dedb413f94489b8009e2db06167a88, user_id=b475d2d866b34290a64141613946dc30, role_id=d0f6ba1314f14d9f9c816bdb5d0c3aac)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:40:45.013210 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db25fe62-2871-4163-94fd-fc2d139fc759 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:40:45.027492 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1207/2383] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 15:40:44 2022] PUT /identity/v3/projects/06dedb413f94489b8009e2db06167a88/users/b475d2d866b34290a64141613946dc30/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 15:40:48.618217 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245a7a3b-fd60-4c58-9170-bdfed122ec95 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:40:48.619158 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245a7a3b-fd60-4c58-9170-bdfed122ec95 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:40:48.619481 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245a7a3b-fd60-4c58-9170-bdfed122ec95 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:40:48.621145 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1177/2384] 10.222.0.99 () {54 vars in 921 bytes} [Thu Jul 14 15:40:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:40:48.635277 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854e0198-515b-447f-a206-b0efdc8e7433 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:40:48.636877 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854e0198-515b-447f-a206-b0efdc8e7433 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:40:48.637350 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854e0198-515b-447f-a206-b0efdc8e7433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:40:48.672026 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-854e0198-515b-447f-a206-b0efdc8e7433 None None] MFA Rules not processed for user `3a9b1c2622934c5185f3b5f1ea623bc0`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:40:48.674246 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-854e0198-515b-447f-a206-b0efdc8e7433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:40:48.888979 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1208/2385] 10.222.0.99 () {58 vars in 1018 bytes} [Thu Jul 14 15:40:48 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 15:40:48.897587 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245a7a3b-fd60-4c58-9170-bdfed122ec95 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:40:48.899867 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-245a7a3b-fd60-4c58-9170-bdfed122ec95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:40:48.909933 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:40:49.037966 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245a7a3b-fd60-4c58-9170-bdfed122ec95 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:40:49.046574 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9142b174-e656-4416-974d-80710d5e9e17 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605e230d50ce44629f554c95b2f79ead', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:40:49.047171 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9142b174-e656-4416-974d-80710d5e9e17 service glance] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:40:49.047546 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9142b174-e656-4416-974d-80710d5e9e17 service glance] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:40:49.048500 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9142b174-e656-4416-974d-80710d5e9e17 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:40:49.081681 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9142b174-e656-4416-974d-80710d5e9e17 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:40:49.085293 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9142b174-e656-4416-974d-80710d5e9e17 service glance] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:40:49.114432 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1178/2386] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:40:48 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:40:49.257330 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053dfba7-df52-422f-b33f-ecf2a77f2ff0 None None] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:40:49.257988 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053dfba7-df52-422f-b33f-ecf2a77f2ff0 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:40:49.259578 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053dfba7-df52-422f-b33f-ecf2a77f2ff0 None None] PATH_INFO: `/` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:40:49.263934 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1209/2387] 10.222.0.99 () {54 vars in 921 bytes} [Thu Jul 14 15:40:49 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:40:49.270653 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaed9e4-d6c1-45af-afc7-d195f1a28cf5 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:40:49.271052 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaed9e4-d6c1-45af-afc7-d195f1a28cf5 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:40:49.271343 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaed9e4-d6c1-45af-afc7-d195f1a28cf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:40:49.302032 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-bfaed9e4-d6c1-45af-afc7-d195f1a28cf5 None None] MFA Rules not processed for user `0d0261759f9743b1a13a760ca9159305`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:40:49.303307 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-bfaed9e4-d6c1-45af-afc7-d195f1a28cf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:40:49.433199 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1179/2388] 10.222.0.99 () {58 vars in 1018 bytes} [Thu Jul 14 15:40:49 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:40:49.447286 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053dfba7-df52-422f-b33f-ecf2a77f2ff0 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:40:49.449240 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-053dfba7-df52-422f-b33f-ecf2a77f2ff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:40:49.473046 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053dfba7-df52-422f-b33f-ecf2a77f2ff0 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:40:49.480502 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98355f20-0f71-4042-b221-d91d3e311789 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:40:49.481343 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98355f20-0f71-4042-b221-d91d3e311789 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:40:49.481723 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98355f20-0f71-4042-b221-d91d3e311789 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:40:49.482071 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98355f20-0f71-4042-b221-d91d3e311789 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:40:49.497292 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98355f20-0f71-4042-b221-d91d3e311789 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:40:49.500326 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98355f20-0f71-4042-b221-d91d3e311789 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:40:49.517573 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1210/2389] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:40:49 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:40:49.728998 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ce4e37-bc59-4d82-8e88-93350e6cea5a None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:40:49.729936 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ce4e37-bc59-4d82-8e88-93350e6cea5a None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:40:49.730438 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ce4e37-bc59-4d82-8e88-93350e6cea5a None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:40:49.733142 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1180/2390] 10.222.0.99 () {54 vars in 921 bytes} [Thu Jul 14 15:40:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:40:49.741896 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b905711a-4ab0-4f93-a72e-b0fd58dd0e90 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:40:49.742751 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b905711a-4ab0-4f93-a72e-b0fd58dd0e90 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:40:49.743241 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b905711a-4ab0-4f93-a72e-b0fd58dd0e90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:40:49.771530 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-b905711a-4ab0-4f93-a72e-b0fd58dd0e90 None None] MFA Rules not processed for user `3a9b1c2622934c5185f3b5f1ea623bc0`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:40:49.774914 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-b905711a-4ab0-4f93-a72e-b0fd58dd0e90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:40:49.794842 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1211/2391] 10.222.0.99 () {58 vars in 1018 bytes} [Thu Jul 14 15:40:49 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:40:49.807531 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ce4e37-bc59-4d82-8e88-93350e6cea5a None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:40:49.825006 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ce4e37-bc59-4d82-8e88-93350e6cea5a None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:40:49.836267 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa5bf18-d5d9-4ea4-b1ae-ba1bc10f1159 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:40:49.836939 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa5bf18-d5d9-4ea4-b1ae-ba1bc10f1159 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:40:49.837231 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa5bf18-d5d9-4ea4-b1ae-ba1bc10f1159 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:40:49.837661 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa5bf18-d5d9-4ea4-b1ae-ba1bc10f1159 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:40:49.851893 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa5bf18-d5d9-4ea4-b1ae-ba1bc10f1159 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:40:49.853891 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa5bf18-d5d9-4ea4-b1ae-ba1bc10f1159 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:40:49.868399 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1181/2392] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:40:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:40:50.546060 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b01f3a-cca4-4125-8640-3458c693f643 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:40:50.546579 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b01f3a-cca4-4125-8640-3458c693f643 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:40:50.547096 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b01f3a-cca4-4125-8640-3458c693f643 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:40:50.570154 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-59b01f3a-cca4-4125-8640-3458c693f643 None None] MFA Rules not processed for user `adaeadf613954d7393b3fd3bae68c362`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:40:50.571446 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-59b01f3a-cca4-4125-8640-3458c693f643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:40:50.621977 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:40:50.676984 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1212/2393] 10.222.0.99 () {58 vars in 1009 bytes} [Thu Jul 14 15:40:50 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:40:50.693587 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa5bf18-d5d9-4ea4-b1ae-ba1bc10f1159 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:40:50.696193 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-6fa5bf18-d5d9-4ea4-b1ae-ba1bc10f1159 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:40:50.796853 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa5bf18-d5d9-4ea4-b1ae-ba1bc10f1159 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:40:50.816602 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84e4ffd-42ed-42d3-b159-975b1755adad service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd2ec3b3ebc4aacb322c95270374451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:40:50.817843 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84e4ffd-42ed-42d3-b159-975b1755adad service swift] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:40:50.818560 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84e4ffd-42ed-42d3-b159-975b1755adad service swift] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:40:50.818925 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84e4ffd-42ed-42d3-b159-975b1755adad service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:40:50.831556 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84e4ffd-42ed-42d3-b159-975b1755adad service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:40:50.833789 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84e4ffd-42ed-42d3-b159-975b1755adad service swift] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:40:50.846274 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1182/2394] 10.222.0.99 () {60 vars in 1384 bytes} [Thu Jul 14 15:40:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 15:40:51.614390 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9541b1c0-b40c-46c7-a313-c16ef42c1c24 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:40:51.614874 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9541b1c0-b40c-46c7-a313-c16ef42c1c24 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:40:51.614997 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9541b1c0-b40c-46c7-a313-c16ef42c1c24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:40:51.643507 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-9541b1c0-b40c-46c7-a313-c16ef42c1c24 None None] MFA Rules not processed for user `adaeadf613954d7393b3fd3bae68c362`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:40:51.645012 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-9541b1c0-b40c-46c7-a313-c16ef42c1c24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:40:51.653712 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1213/2395] 10.222.0.99 () {58 vars in 967 bytes} [Thu Jul 14 15:40:51 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:40:51.669792 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84e4ffd-42ed-42d3-b159-975b1755adad service swift] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:40:51.682426 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84e4ffd-42ed-42d3-b159-975b1755adad service swift] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:40:51.689412 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8388f4ac-2252-4010-a792-8f1212b5dfe1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd2ec3b3ebc4aacb322c95270374451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:40:51.690070 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8388f4ac-2252-4010-a792-8f1212b5dfe1 service swift] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:40:51.690352 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8388f4ac-2252-4010-a792-8f1212b5dfe1 service swift] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:40:51.690628 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8388f4ac-2252-4010-a792-8f1212b5dfe1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:40:51.699853 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8388f4ac-2252-4010-a792-8f1212b5dfe1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:40:51.701456 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8388f4ac-2252-4010-a792-8f1212b5dfe1 service swift] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:40:51.712357 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1183/2396] 10.222.0.99 () {60 vars in 1384 bytes} [Thu Jul 14 15:40:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 15:40:52.200812 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7afb1e-20b2-4664-b900-101a12e0fbfe None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:40:52.202201 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7afb1e-20b2-4664-b900-101a12e0fbfe None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:40:52.202639 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7afb1e-20b2-4664-b900-101a12e0fbfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:40:52.226167 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-8b7afb1e-20b2-4664-b900-101a12e0fbfe None None] MFA Rules not processed for user `adaeadf613954d7393b3fd3bae68c362`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:40:52.229550 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-8b7afb1e-20b2-4664-b900-101a12e0fbfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:40:52.238742 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1214/2397] 10.222.0.99 () {58 vars in 1009 bytes} [Thu Jul 14 15:40:52 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:40:52.253146 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8388f4ac-2252-4010-a792-8f1212b5dfe1 service swift] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:40:52.264779 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8388f4ac-2252-4010-a792-8f1212b5dfe1 service swift] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:40:52.271880 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3cd18c-4ab7-4a0b-b310-b3a688f81c79 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd2ec3b3ebc4aacb322c95270374451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:40:52.272542 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3cd18c-4ab7-4a0b-b310-b3a688f81c79 service swift] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:40:52.272829 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3cd18c-4ab7-4a0b-b310-b3a688f81c79 service swift] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:40:52.273951 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3cd18c-4ab7-4a0b-b310-b3a688f81c79 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:40:52.284744 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3cd18c-4ab7-4a0b-b310-b3a688f81c79 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:40:52.287073 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3cd18c-4ab7-4a0b-b310-b3a688f81c79 service swift] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:40:52.299478 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1184/2398] 10.222.0.99 () {60 vars in 1384 bytes} [Thu Jul 14 15:40:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 15:40:52.414488 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b11d0a-499b-4f0a-b2ed-0d83a23fce8f None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:40:52.414997 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b11d0a-499b-4f0a-b2ed-0d83a23fce8f None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:40:52.415436 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b11d0a-499b-4f0a-b2ed-0d83a23fce8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:40:52.466870 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-c6b11d0a-499b-4f0a-b2ed-0d83a23fce8f None None] MFA Rules not processed for user `b475d2d866b34290a64141613946dc30`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:40:52.470830 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-c6b11d0a-499b-4f0a-b2ed-0d83a23fce8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:40:52.695714 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1215/2399] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:40:52 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:40:52.710027 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3cd18c-4ab7-4a0b-b310-b3a688f81c79 service swift] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:40:52.711540 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0d3cd18c-4ab7-4a0b-b310-b3a688f81c79 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:40:52.727730 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:40:52.732642 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3cd18c-4ab7-4a0b-b310-b3a688f81c79 service swift] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:40:52.739637 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c898c83-d176-423e-aa53-3b2e95166675 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9b1c2622934c5185f3b5f1ea623bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:40:52.740531 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c898c83-d176-423e-aa53-3b2e95166675 service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:40:52.740818 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c898c83-d176-423e-aa53-3b2e95166675 service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:40:52.741345 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c898c83-d176-423e-aa53-3b2e95166675 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:40:52.821362 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c898c83-d176-423e-aa53-3b2e95166675 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:40:52.823178 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c898c83-d176-423e-aa53-3b2e95166675 service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:40:52.836238 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1185/2400] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:40:52 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:40:53.140202 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e11474f-436c-469a-97f2-c8f77ce71446 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:40:53.141986 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e11474f-436c-469a-97f2-c8f77ce71446 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:40:53.142243 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e11474f-436c-469a-97f2-c8f77ce71446 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:40:53.164301 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-0e11474f-436c-469a-97f2-c8f77ce71446 None None] MFA Rules not processed for user `adaeadf613954d7393b3fd3bae68c362`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:40:53.166368 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-0e11474f-436c-469a-97f2-c8f77ce71446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:40:53.177144 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1216/2401] 10.222.0.99 () {58 vars in 967 bytes} [Thu Jul 14 15:40:53 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:40:53.210390 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c898c83-d176-423e-aa53-3b2e95166675 service nova] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:40:53.227250 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c898c83-d176-423e-aa53-3b2e95166675 service nova] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:40:53.242923 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7ed036-eff9-43c1-a59a-6531ad7f8a33 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd2ec3b3ebc4aacb322c95270374451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:40:53.244058 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7ed036-eff9-43c1-a59a-6531ad7f8a33 service swift] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:40:53.244866 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7ed036-eff9-43c1-a59a-6531ad7f8a33 service swift] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:40:53.245381 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7ed036-eff9-43c1-a59a-6531ad7f8a33 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:40:53.259311 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7ed036-eff9-43c1-a59a-6531ad7f8a33 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:40:53.261193 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7ed036-eff9-43c1-a59a-6531ad7f8a33 service swift] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:40:53.275864 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1186/2402] 10.222.0.99 () {60 vars in 1384 bytes} [Thu Jul 14 15:40:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 15:40:54.251123 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da3f476-0026-45f4-8d52-b3cd2a6b57a9 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:40:54.251123 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da3f476-0026-45f4-8d52-b3cd2a6b57a9 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:40:54.251123 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da3f476-0026-45f4-8d52-b3cd2a6b57a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:40:54.273382 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-0da3f476-0026-45f4-8d52-b3cd2a6b57a9 None None] MFA Rules not processed for user `dbeb3476ffee4611896156633c84265c`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:40:54.274887 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-0da3f476-0026-45f4-8d52-b3cd2a6b57a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:40:54.282633 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1217/2403] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:40:54 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:40:58.723187 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7ed036-eff9-43c1-a59a-6531ad7f8a33 service swift] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:40:58.724886 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-6a7ed036-eff9-43c1-a59a-6531ad7f8a33 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:40:58.810317 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7ed036-eff9-43c1-a59a-6531ad7f8a33 service swift] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:40:58.817263 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c36d24-4f37-48b4-a395-c6f5e5e006d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:40:58.819077 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c36d24-4f37-48b4-a395-c6f5e5e006d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:40:58.819777 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c36d24-4f37-48b4-a395-c6f5e5e006d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:40:58.820323 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c36d24-4f37-48b4-a395-c6f5e5e006d3 admin admin] PATH_INFO: `/v3/users/4689786287394deb9ecafb9f2cdbca89` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:40:58.821977 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c36d24-4f37-48b4-a395-c6f5e5e006d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4689786287394deb9ecafb9f2cdbca89)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:40:58.824454 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c36d24-4f37-48b4-a395-c6f5e5e006d3 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:40:58.879794 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-92c36d24-4f37-48b4-a395-c6f5e5e006d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4689786287394deb9ecafb9f2cdbca89'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:40:58.887380 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-92c36d24-4f37-48b4-a395-c6f5e5e006d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4689786287394deb9ecafb9f2cdbca89'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:40:58.904456 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-92c36d24-4f37-48b4-a395-c6f5e5e006d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4689786287394deb9ecafb9f2cdbca89'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:40:58.905297 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-92c36d24-4f37-48b4-a395-c6f5e5e006d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4689786287394deb9ecafb9f2cdbca89'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:40:58.906011 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-92c36d24-4f37-48b4-a395-c6f5e5e006d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4689786287394deb9ecafb9f2cdbca89'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:40:58.921112 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1187/2404] 10.222.0.99 () {58 vars in 1223 bytes} [Thu Jul 14 15:40:58 2022] DELETE /identity/v3/users/4689786287394deb9ecafb9f2cdbca89 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:40:58.925990 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8cb0eb-1925-4607-8d4c-bf9feef810ee None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:40:58.926886 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8cb0eb-1925-4607-8d4c-bf9feef810ee None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:40:58.927333 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8cb0eb-1925-4607-8d4c-bf9feef810ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:40:58.971274 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-0a8cb0eb-1925-4607-8d4c-bf9feef810ee None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:40:58.972635 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-0a8cb0eb-1925-4607-8d4c-bf9feef810ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:40:59.047628 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1218/2405] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:40:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:40:59.057450 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c36d24-4f37-48b4-a395-c6f5e5e006d3 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:40:59.058872 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-92c36d24-4f37-48b4-a395-c6f5e5e006d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:40:59.071204 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:40:59.125071 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c36d24-4f37-48b4-a395-c6f5e5e006d3 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:40:59.130743 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9786d3c2-6b8f-4655-8399-5918287aec40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:40:59.131847 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9786d3c2-6b8f-4655-8399-5918287aec40 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:40:59.132241 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9786d3c2-6b8f-4655-8399-5918287aec40 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:40:59.132735 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9786d3c2-6b8f-4655-8399-5918287aec40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:40:59.144931 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9786d3c2-6b8f-4655-8399-5918287aec40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:40:59.146536 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9786d3c2-6b8f-4655-8399-5918287aec40 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:40:59.156998 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1188/2406] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:40:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:40:59.426182 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8cb0eb-1925-4607-8d4c-bf9feef810ee None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:40:59.427655 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-0a8cb0eb-1925-4607-8d4c-bf9feef810ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:40:59.441200 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8cb0eb-1925-4607-8d4c-bf9feef810ee None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:40:59.448993 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab52e0ef-d2d3-47a6-b064-8bf87b8fa3b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:40:59.450133 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab52e0ef-d2d3-47a6-b064-8bf87b8fa3b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:40:59.450430 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab52e0ef-d2d3-47a6-b064-8bf87b8fa3b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:40:59.450716 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab52e0ef-d2d3-47a6-b064-8bf87b8fa3b9 admin admin] PATH_INFO: `/v3/projects/75292a36c8354a2dbd28428efb2e9df9` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:40:59.452000 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab52e0ef-d2d3-47a6-b064-8bf87b8fa3b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=75292a36c8354a2dbd28428efb2e9df9)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:40:59.454547 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab52e0ef-d2d3-47a6-b064-8bf87b8fa3b9 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:40:59.499887 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:40:59.516925 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-ab52e0ef-d2d3-47a6-b064-8bf87b8fa3b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75292a36c8354a2dbd28428efb2e9df9', 'request_id': 'req-ab52e0ef-d2d3-47a6-b064-8bf87b8fa3b9'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:40:59.525397 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-ab52e0ef-d2d3-47a6-b064-8bf87b8fa3b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75292a36c8354a2dbd28428efb2e9df9', 'request_id': 'req-ab52e0ef-d2d3-47a6-b064-8bf87b8fa3b9'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:40:59.533137 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-ab52e0ef-d2d3-47a6-b064-8bf87b8fa3b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75292a36c8354a2dbd28428efb2e9df9', 'request_id': 'req-ab52e0ef-d2d3-47a6-b064-8bf87b8fa3b9'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:40:59.551905 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-ab52e0ef-d2d3-47a6-b064-8bf87b8fa3b9 admin admin] The token cache is being invalidate because project 75292a36c8354a2dbd28428efb2e9df9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 15:40:59.552554 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-ab52e0ef-d2d3-47a6-b064-8bf87b8fa3b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:40:59.553217 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-ab52e0ef-d2d3-47a6-b064-8bf87b8fa3b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:40:59.554990 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1219/2407] 10.222.0.99 () {58 vars in 1232 bytes} [Thu Jul 14 15:40:59 2022] DELETE /identity/v3/projects/75292a36c8354a2dbd28428efb2e9df9 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:40:59.984929 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e90277f-fcec-45a0-be17-c1fdcf85cb53 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:40:59.986010 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e90277f-fcec-45a0-be17-c1fdcf85cb53 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:40:59.986010 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e90277f-fcec-45a0-be17-c1fdcf85cb53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:41:00.007371 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-0e90277f-fcec-45a0-be17-c1fdcf85cb53 None None] MFA Rules not processed for user `adaeadf613954d7393b3fd3bae68c362`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:41:00.008659 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0e90277f-fcec-45a0-be17-c1fdcf85cb53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:41:00.073024 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1189/2408] 10.222.0.99 () {58 vars in 1009 bytes} [Thu Jul 14 15:40:59 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:41:00.085562 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab52e0ef-d2d3-47a6-b064-8bf87b8fa3b9 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:41:00.086986 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-ab52e0ef-d2d3-47a6-b064-8bf87b8fa3b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:41:00.167761 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab52e0ef-d2d3-47a6-b064-8bf87b8fa3b9 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:41:00.180715 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd42e88-ce33-4692-9aa8-2715392811e9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd2ec3b3ebc4aacb322c95270374451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:41:00.180715 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd42e88-ce33-4692-9aa8-2715392811e9 service swift] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:41:00.180715 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd42e88-ce33-4692-9aa8-2715392811e9 service swift] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:41:00.180715 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd42e88-ce33-4692-9aa8-2715392811e9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:41:00.184968 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd42e88-ce33-4692-9aa8-2715392811e9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:41:00.186612 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd42e88-ce33-4692-9aa8-2715392811e9 service swift] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:41:00.198392 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1220/2409] 10.222.0.99 () {60 vars in 1384 bytes} [Thu Jul 14 15:41:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 15:41:05.848141 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266f4167-008b-4965-8b0f-a6c0e38a1933 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:41:05.848628 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266f4167-008b-4965-8b0f-a6c0e38a1933 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:41:05.848722 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266f4167-008b-4965-8b0f-a6c0e38a1933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:41:05.868160 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-266f4167-008b-4965-8b0f-a6c0e38a1933 None None] MFA Rules not processed for user `71bc270c6253426baa4f79638d466b20`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:41:05.869380 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-266f4167-008b-4965-8b0f-a6c0e38a1933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:41:05.949512 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1190/2410] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:41:05 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:41:10.193368 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=44,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:41:15.941915 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=72 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:41:22.818074 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd42e88-ce33-4692-9aa8-2715392811e9 service swift] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:41:22.819580 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-7dd42e88-ce33-4692-9aa8-2715392811e9 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:41:22.886850 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd42e88-ce33-4692-9aa8-2715392811e9 service swift] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:41:22.891974 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e502777-9136-411b-9cd1-cdc286439061 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:41:22.892636 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e502777-9136-411b-9cd1-cdc286439061 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:41:22.893045 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e502777-9136-411b-9cd1-cdc286439061 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:41:22.893204 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e502777-9136-411b-9cd1-cdc286439061 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:41:22.896041 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-9e502777-9136-411b-9cd1-cdc286439061 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:41:22.979842 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e502777-9136-411b-9cd1-cdc286439061 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:41:22.982386 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e502777-9136-411b-9cd1-cdc286439061 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:41:23.001414 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1221/2411] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:41:22 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:41:30.063871 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266f4167-008b-4965-8b0f-a6c0e38a1933 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:41:30.077569 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266f4167-008b-4965-8b0f-a6c0e38a1933 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:41:30.083008 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07017510-8375-4959-8dba-4b433d116be9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:41:30.083705 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07017510-8375-4959-8dba-4b433d116be9 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:41:30.084000 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07017510-8375-4959-8dba-4b433d116be9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:41:30.084284 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07017510-8375-4959-8dba-4b433d116be9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:41:30.086325 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-07017510-8375-4959-8dba-4b433d116be9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:41:30.162005 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07017510-8375-4959-8dba-4b433d116be9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:41:30.163455 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07017510-8375-4959-8dba-4b433d116be9 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:41:30.174710 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1191/2412] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:41:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:41:31.607852 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e502777-9136-411b-9cd1-cdc286439061 service neutron] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:41:31.609237 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-9e502777-9136-411b-9cd1-cdc286439061 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:41:31.620767 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e502777-9136-411b-9cd1-cdc286439061 service neutron] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:41:31.624352 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2fbf8b-8aa9-4287-82dd-bf8ef9987698 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:41:31.624998 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2fbf8b-8aa9-4287-82dd-bf8ef9987698 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:41:31.625284 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2fbf8b-8aa9-4287-82dd-bf8ef9987698 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:41:31.625564 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2fbf8b-8aa9-4287-82dd-bf8ef9987698 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:41:31.627275 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-dd2fbf8b-8aa9-4287-82dd-bf8ef9987698 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:41:31.641253 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2fbf8b-8aa9-4287-82dd-bf8ef9987698 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:41:31.642687 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2fbf8b-8aa9-4287-82dd-bf8ef9987698 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:41:31.655660 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1222/2413] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:41:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:41:40.169711 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=34 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:41:41.650150 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=56 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:41:42.817989 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd6b6fa-f4d0-4e26-8039-8df8d0d4f46b None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:41:42.818483 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd6b6fa-f4d0-4e26-8039-8df8d0d4f46b None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:41:42.818579 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd6b6fa-f4d0-4e26-8039-8df8d0d4f46b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:41:42.836677 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-7cd6b6fa-f4d0-4e26-8039-8df8d0d4f46b None None] MFA Rules not processed for user `adaeadf613954d7393b3fd3bae68c362`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:41:42.837924 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-7cd6b6fa-f4d0-4e26-8039-8df8d0d4f46b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:41:42.845207 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1192/2414] 10.222.0.99 () {58 vars in 967 bytes} [Thu Jul 14 15:41:42 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:41:42.869389 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2fbf8b-8aa9-4287-82dd-bf8ef9987698 service neutron] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:41:42.880569 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2fbf8b-8aa9-4287-82dd-bf8ef9987698 service neutron] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:41:42.885696 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185bdef7-8923-4c66-8814-ea28c7af5c2b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd2ec3b3ebc4aacb322c95270374451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:41:42.886344 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185bdef7-8923-4c66-8814-ea28c7af5c2b service swift] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:41:42.886636 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185bdef7-8923-4c66-8814-ea28c7af5c2b service swift] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:41:42.886918 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185bdef7-8923-4c66-8814-ea28c7af5c2b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:41:42.896121 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185bdef7-8923-4c66-8814-ea28c7af5c2b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:41:42.897734 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185bdef7-8923-4c66-8814-ea28c7af5c2b service swift] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:41:42.906772 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1223/2415] 10.222.0.99 () {60 vars in 1384 bytes} [Thu Jul 14 15:41:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 15:41:43.652489 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd6b6fa-f4d0-4e26-8039-8df8d0d4f46b None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:41:43.662338 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd6b6fa-f4d0-4e26-8039-8df8d0d4f46b None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:41:43.665881 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694a3b35-096f-4a04-b384-ecdac933e217 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:41:43.666530 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694a3b35-096f-4a04-b384-ecdac933e217 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:41:43.666814 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694a3b35-096f-4a04-b384-ecdac933e217 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:41:43.667091 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694a3b35-096f-4a04-b384-ecdac933e217 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:41:43.669059 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-694a3b35-096f-4a04-b384-ecdac933e217 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:41:43.735752 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694a3b35-096f-4a04-b384-ecdac933e217 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:41:43.737252 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694a3b35-096f-4a04-b384-ecdac933e217 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:41:43.748268 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1193/2416] 10.222.0.99 () {60 vars in 1396 bytes} [Thu Jul 14 15:41:43 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3256 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:41:49.953163 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185bdef7-8923-4c66-8814-ea28c7af5c2b service swift] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:41:49.962938 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185bdef7-8923-4c66-8814-ea28c7af5c2b service swift] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:41:49.966817 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd15b32-981b-4867-8356-bf614662ae0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:41:49.967453 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd15b32-981b-4867-8356-bf614662ae0e service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:41:49.967779 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd15b32-981b-4867-8356-bf614662ae0e service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:41:49.968065 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd15b32-981b-4867-8356-bf614662ae0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:41:49.969848 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-bbd15b32-981b-4867-8356-bf614662ae0e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:41:50.040342 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd15b32-981b-4867-8356-bf614662ae0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:41:50.041808 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd15b32-981b-4867-8356-bf614662ae0e service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:41:50.051480 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1224/2417] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:41:49 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:41:51.773939 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694a3b35-096f-4a04-b384-ecdac933e217 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:41:51.775701 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-694a3b35-096f-4a04-b384-ecdac933e217 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:41:51.791628 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694a3b35-096f-4a04-b384-ecdac933e217 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:41:51.798411 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d1b65a-9a63-408e-a8b6-2b3e106b001e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9b1c2622934c5185f3b5f1ea623bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:41:51.799060 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d1b65a-9a63-408e-a8b6-2b3e106b001e service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:41:51.799587 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d1b65a-9a63-408e-a8b6-2b3e106b001e service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:41:51.800202 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d1b65a-9a63-408e-a8b6-2b3e106b001e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:41:51.802408 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-99d1b65a-9a63-408e-a8b6-2b3e106b001e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:41:51.841274 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd15b32-981b-4867-8356-bf614662ae0e service neutron] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:41:51.849539 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd15b32-981b-4867-8356-bf614662ae0e service neutron] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:41:51.854102 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fc8c5d-57e2-4635-b507-d293fa96e3d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:41:51.854764 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fc8c5d-57e2-4635-b507-d293fa96e3d9 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:41:51.855053 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fc8c5d-57e2-4635-b507-d293fa96e3d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:41:51.855331 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fc8c5d-57e2-4635-b507-d293fa96e3d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:41:51.857732 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-e9fc8c5d-57e2-4635-b507-d293fa96e3d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:41:51.868484 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d1b65a-9a63-408e-a8b6-2b3e106b001e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:41:51.870123 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d1b65a-9a63-408e-a8b6-2b3e106b001e service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:41:51.870506 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fc8c5d-57e2-4635-b507-d293fa96e3d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:41:51.872004 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fc8c5d-57e2-4635-b507-d293fa96e3d9 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:41:51.880949 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1194/2418] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:41:51 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:41:51.883013 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1225/2419] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:41:51 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 15:42:01.875226 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=59 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:42:01.878718 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=42 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:42:18.038125 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d1b65a-9a63-408e-a8b6-2b3e106b001e service nova] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:42:18.039660 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-99d1b65a-9a63-408e-a8b6-2b3e106b001e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:42:18.054872 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d1b65a-9a63-408e-a8b6-2b3e106b001e service nova] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:42:18.061068 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6315d785-b296-4606-8972-d1688f14610c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9b1c2622934c5185f3b5f1ea623bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:42:18.061728 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6315d785-b296-4606-8972-d1688f14610c service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:42:18.062014 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6315d785-b296-4606-8972-d1688f14610c service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:42:18.062293 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6315d785-b296-4606-8972-d1688f14610c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:42:18.064075 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-6315d785-b296-4606-8972-d1688f14610c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:42:18.127254 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6315d785-b296-4606-8972-d1688f14610c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:42:18.128901 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6315d785-b296-4606-8972-d1688f14610c service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:42:18.138544 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1195/2420] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:42:18 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 15:42:28.133862 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=28 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:43:19.893104 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd06c16-a1d2-43c3-a851-a050e2add926 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:43:19.894289 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd06c16-a1d2-43c3-a851-a050e2add926 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:43:19.894289 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd06c16-a1d2-43c3-a851-a050e2add926 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:43:19.911704 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-8cd06c16-a1d2-43c3-a851-a050e2add926 None None] MFA Rules not processed for user `d01c301ea20d4acab8dff1148501a975`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:43:19.912923 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-8cd06c16-a1d2-43c3-a851-a050e2add926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:43:19.919985 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1226/2421] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 15:43:19 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:43:24.906526 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6315d785-b296-4606-8972-d1688f14610c service nova] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:43:24.919927 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6315d785-b296-4606-8972-d1688f14610c service nova] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:43:24.932378 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba695c7-cbdc-4400-9662-2aafecb75146 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9b1c2622934c5185f3b5f1ea623bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:43:24.932378 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba695c7-cbdc-4400-9662-2aafecb75146 service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:43:24.932378 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba695c7-cbdc-4400-9662-2aafecb75146 service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:43:24.932378 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba695c7-cbdc-4400-9662-2aafecb75146 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:43:24.936428 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-6ba695c7-cbdc-4400-9662-2aafecb75146 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:43:24.948357 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba695c7-cbdc-4400-9662-2aafecb75146 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:43:24.949970 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba695c7-cbdc-4400-9662-2aafecb75146 service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:43:24.962640 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1196/2422] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:43:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:43:29.906865 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=3 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:43:34.957106 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=6 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:43:39.627218 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd06c16-a1d2-43c3-a851-a050e2add926 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:43:39.628756 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-8cd06c16-a1d2-43c3-a851-a050e2add926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:43:39.687370 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd06c16-a1d2-43c3-a851-a050e2add926 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:43:39.691247 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6f5b0d-1684-4b6f-a9a7-1652277fde23 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3e1a2b83f3d4d88ba5084dfaffd22f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:43:39.691938 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6f5b0d-1684-4b6f-a9a7-1652277fde23 service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:43:39.692261 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6f5b0d-1684-4b6f-a9a7-1652277fde23 service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:43:39.692550 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6f5b0d-1684-4b6f-a9a7-1652277fde23 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:43:39.694330 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-cd6f5b0d-1684-4b6f-a9a7-1652277fde23 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:43:39.705312 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6f5b0d-1684-4b6f-a9a7-1652277fde23 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:43:39.706750 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6f5b0d-1684-4b6f-a9a7-1652277fde23 service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:43:39.715864 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1227/2423] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:43:39 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:43:49.711821 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=26 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:44:12.066661 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba695c7-cbdc-4400-9662-2aafecb75146 service nova] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:44:12.077921 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba695c7-cbdc-4400-9662-2aafecb75146 service nova] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:44:12.083356 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdded4a-d98b-437e-a42f-09ee42828a8f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9b1c2622934c5185f3b5f1ea623bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:44:12.084021 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdded4a-d98b-437e-a42f-09ee42828a8f service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:44:12.084313 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdded4a-d98b-437e-a42f-09ee42828a8f service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:44:12.084590 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdded4a-d98b-437e-a42f-09ee42828a8f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:44:12.086975 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-3cdded4a-d98b-437e-a42f-09ee42828a8f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:44:12.145470 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdded4a-d98b-437e-a42f-09ee42828a8f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:44:12.147072 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdded4a-d98b-437e-a42f-09ee42828a8f service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:44:12.157395 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1197/2424] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:44:12 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:44:22.153049 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=28 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:44:30.877947 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6f5b0d-1684-4b6f-a9a7-1652277fde23 service cinder] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:44:30.879553 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-cd6f5b0d-1684-4b6f-a9a7-1652277fde23 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:44:31.037993 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6f5b0d-1684-4b6f-a9a7-1652277fde23 service cinder] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:44:31.043049 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14241eef-7ec9-420f-b73d-eb020b0f61a8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc157c485f794e968a60d8405c9310ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:44:31.043728 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14241eef-7ec9-420f-b73d-eb020b0f61a8 service placement] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:44:31.044019 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14241eef-7ec9-420f-b73d-eb020b0f61a8 service placement] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:44:31.044300 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14241eef-7ec9-420f-b73d-eb020b0f61a8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:44:31.047226 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-14241eef-7ec9-420f-b73d-eb020b0f61a8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:44:31.058984 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14241eef-7ec9-420f-b73d-eb020b0f61a8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:44:31.061711 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14241eef-7ec9-420f-b73d-eb020b0f61a8 service placement] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:44:31.071688 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1228/2425] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:44:30 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:44:41.067344 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=63 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:44:44.481686 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdded4a-d98b-437e-a42f-09ee42828a8f service nova] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:44:44.552743 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdded4a-d98b-437e-a42f-09ee42828a8f service nova] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:44:44.558907 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d1330d-6341-426e-b17a-6eb856f37baa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9b1c2622934c5185f3b5f1ea623bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:44:44.559565 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d1330d-6341-426e-b17a-6eb856f37baa service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:44:44.559888 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d1330d-6341-426e-b17a-6eb856f37baa service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:44:44.560173 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d1330d-6341-426e-b17a-6eb856f37baa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:44:44.562981 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-85d1330d-6341-426e-b17a-6eb856f37baa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:44:44.588064 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d1330d-6341-426e-b17a-6eb856f37baa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:44:44.589670 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d1330d-6341-426e-b17a-6eb856f37baa service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:44:44.602653 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1198/2426] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:44:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:44:54.597653 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=32 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:45:53.652720 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14241eef-7ec9-420f-b73d-eb020b0f61a8 service placement] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:45:53.666540 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14241eef-7ec9-420f-b73d-eb020b0f61a8 service placement] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:45:53.672138 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2907f3b-e7ca-44ec-9c43-b057f7e0431f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9b1c2622934c5185f3b5f1ea623bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:45:53.673501 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2907f3b-e7ca-44ec-9c43-b057f7e0431f service nova] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:45:53.673790 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2907f3b-e7ca-44ec-9c43-b057f7e0431f service nova] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:45:53.674214 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2907f3b-e7ca-44ec-9c43-b057f7e0431f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:45:53.676129 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-f2907f3b-e7ca-44ec-9c43-b057f7e0431f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:45:53.736360 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2907f3b-e7ca-44ec-9c43-b057f7e0431f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:45:53.737407 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2907f3b-e7ca-44ec-9c43-b057f7e0431f service nova] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:45:53.746358 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1229/2427] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:45:53 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 15:46:03.742333 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=28 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:47:33.419151 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d1330d-6341-426e-b17a-6eb856f37baa service nova] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:47:33.439272 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d1330d-6341-426e-b17a-6eb856f37baa service nova] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:47:33.443649 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af83290-226f-434d-8e6b-4c12031d8280 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:47:33.444311 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af83290-226f-434d-8e6b-4c12031d8280 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:47:33.444596 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af83290-226f-434d-8e6b-4c12031d8280 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:47:33.444876 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af83290-226f-434d-8e6b-4c12031d8280 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:47:33.452661 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af83290-226f-434d-8e6b-4c12031d8280 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:47:33.454144 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af83290-226f-434d-8e6b-4c12031d8280 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:47:33.462949 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1199/2428] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:47:33 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:47:43.459125 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=10 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:48:01.593581 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2907f3b-e7ca-44ec-9c43-b057f7e0431f service nova] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:48:01.612879 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2907f3b-e7ca-44ec-9c43-b057f7e0431f service nova] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:48:01.617268 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa53b6b-f200-4076-9587-90521f005b5a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3e1a2b83f3d4d88ba5084dfaffd22f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:48:01.617915 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa53b6b-f200-4076-9587-90521f005b5a service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:48:01.618203 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa53b6b-f200-4076-9587-90521f005b5a service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:48:01.618484 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa53b6b-f200-4076-9587-90521f005b5a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:48:01.629471 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa53b6b-f200-4076-9587-90521f005b5a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:48:01.631634 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa53b6b-f200-4076-9587-90521f005b5a service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:48:01.642931 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1230/2429] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:48:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:48:01.650299 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af83290-226f-434d-8e6b-4c12031d8280 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:48:01.659774 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af83290-226f-434d-8e6b-4c12031d8280 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:48:01.663951 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1924c847-e762-474a-8f74-af18c841eafb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3e1a2b83f3d4d88ba5084dfaffd22f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:48:01.664588 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1924c847-e762-474a-8f74-af18c841eafb service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:48:01.664885 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1924c847-e762-474a-8f74-af18c841eafb service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:48:01.665177 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1924c847-e762-474a-8f74-af18c841eafb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:48:01.673829 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1924c847-e762-474a-8f74-af18c841eafb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:48:01.675260 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1924c847-e762-474a-8f74-af18c841eafb service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:48:01.705603 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1200/2430] 10.222.0.99 () {60 vars in 1396 bytes} [Thu Jul 14 15:48:01 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3260 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 15:48:01.751513 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa53b6b-f200-4076-9587-90521f005b5a service cinder] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:48:01.762625 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa53b6b-f200-4076-9587-90521f005b5a service cinder] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:48:01.766484 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eafc47-8c23-4c5e-92cc-97f1f303f894 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3e1a2b83f3d4d88ba5084dfaffd22f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:48:01.767133 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eafc47-8c23-4c5e-92cc-97f1f303f894 service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:48:01.767476 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eafc47-8c23-4c5e-92cc-97f1f303f894 service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:48:01.767733 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eafc47-8c23-4c5e-92cc-97f1f303f894 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:48:01.781446 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00eafc47-8c23-4c5e-92cc-97f1f303f894 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:48:01.783677 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00eafc47-8c23-4c5e-92cc-97f1f303f894 service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:48:01.798424 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1231/2431] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:48:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:48:01.844620 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1924c847-e762-474a-8f74-af18c841eafb service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:48:01.861417 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1924c847-e762-474a-8f74-af18c841eafb service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:48:01.862866 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bf178f-da8d-4b7d-9db4-434069c4690c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3e1a2b83f3d4d88ba5084dfaffd22f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:48:01.863700 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bf178f-da8d-4b7d-9db4-434069c4690c service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:48:01.863960 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bf178f-da8d-4b7d-9db4-434069c4690c service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:48:01.864770 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bf178f-da8d-4b7d-9db4-434069c4690c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:48:01.880873 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bf178f-da8d-4b7d-9db4-434069c4690c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:48:01.881753 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bf178f-da8d-4b7d-9db4-434069c4690c service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:48:01.924013 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1201/2432] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:48:01 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:48:01.937262 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eafc47-8c23-4c5e-92cc-97f1f303f894 service cinder] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:48:01.949325 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eafc47-8c23-4c5e-92cc-97f1f303f894 service cinder] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:48:01.955578 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d355e963-8c42-449f-bdcc-a23a9768d7b4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9b1c2622934c5185f3b5f1ea623bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:48:01.956258 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d355e963-8c42-449f-bdcc-a23a9768d7b4 service nova] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:48:01.956546 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d355e963-8c42-449f-bdcc-a23a9768d7b4 service nova] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:48:01.956822 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d355e963-8c42-449f-bdcc-a23a9768d7b4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:48:01.966457 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d355e963-8c42-449f-bdcc-a23a9768d7b4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:48:01.974806 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d355e963-8c42-449f-bdcc-a23a9768d7b4 service nova] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:48:02.018055 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1232/2433] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:48:01 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:48:11.920197 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=26 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:48:12.013830 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=29 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:48:58.980016 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bf178f-da8d-4b7d-9db4-434069c4690c service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:48:58.981426 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a0bf178f-da8d-4b7d-9db4-434069c4690c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:48:59.010496 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bf178f-da8d-4b7d-9db4-434069c4690c service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:48:59.014349 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84b5f17-5168-4cd8-9275-212011827f67 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3e1a2b83f3d4d88ba5084dfaffd22f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:48:59.015113 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84b5f17-5168-4cd8-9275-212011827f67 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:48:59.015402 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84b5f17-5168-4cd8-9275-212011827f67 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:48:59.015707 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84b5f17-5168-4cd8-9275-212011827f67 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:48:59.024705 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84b5f17-5168-4cd8-9275-212011827f67 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:48:59.026381 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84b5f17-5168-4cd8-9275-212011827f67 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:48:59.053882 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1202/2434] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:48:58 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:49:09.051237 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=18 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:49:35.848721 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d355e963-8c42-449f-bdcc-a23a9768d7b4 service nova] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:49:35.850119 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-d355e963-8c42-449f-bdcc-a23a9768d7b4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:49:35.865345 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d355e963-8c42-449f-bdcc-a23a9768d7b4 service nova] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:49:35.870185 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb749a04-d88b-4f4d-9b1a-bf365980d92a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc157c485f794e968a60d8405c9310ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:49:35.871618 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb749a04-d88b-4f4d-9b1a-bf365980d92a service placement] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:49:35.872132 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb749a04-d88b-4f4d-9b1a-bf365980d92a service placement] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:49:35.872417 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb749a04-d88b-4f4d-9b1a-bf365980d92a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:49:35.880833 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb749a04-d88b-4f4d-9b1a-bf365980d92a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:49:35.882421 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb749a04-d88b-4f4d-9b1a-bf365980d92a service placement] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:49:35.893058 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1233/2435] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:49:35 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 15:49:45.889003 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=6 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:52:02.321125 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84b5f17-5168-4cd8-9275-212011827f67 service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:52:02.360542 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb749a04-d88b-4f4d-9b1a-bf365980d92a service placement] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:52:02.362126 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-eb749a04-d88b-4f4d-9b1a-bf365980d92a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:52:02.415516 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84b5f17-5168-4cd8-9275-212011827f67 service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:52:02.421115 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f7d7d0-2dca-4623-8929-b630c50c8160 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9b1c2622934c5185f3b5f1ea623bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:52:02.421774 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f7d7d0-2dca-4623-8929-b630c50c8160 service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:52:02.422055 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f7d7d0-2dca-4623-8929-b630c50c8160 service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:52:02.422352 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f7d7d0-2dca-4623-8929-b630c50c8160 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:52:02.431312 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f7d7d0-2dca-4623-8929-b630c50c8160 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:52:02.432991 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f7d7d0-2dca-4623-8929-b630c50c8160 service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:52:02.447357 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb749a04-d88b-4f4d-9b1a-bf365980d92a service placement] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:52:02.452991 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729d76ca-72d7-440f-b87b-a2b5cfff483d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9b1c2622934c5185f3b5f1ea623bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:52:02.453804 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729d76ca-72d7-440f-b87b-a2b5cfff483d service nova] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:52:02.454214 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729d76ca-72d7-440f-b87b-a2b5cfff483d service nova] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:52:02.454593 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729d76ca-72d7-440f-b87b-a2b5cfff483d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:52:02.461125 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1203/2436] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:52:02 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:52:02.462066 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f7d7d0-2dca-4623-8929-b630c50c8160 service nova] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:52:02.464514 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729d76ca-72d7-440f-b87b-a2b5cfff483d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:52:02.466225 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729d76ca-72d7-440f-b87b-a2b5cfff483d service nova] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:52:02.471560 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f7d7d0-2dca-4623-8929-b630c50c8160 service nova] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:52:02.475589 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9fa4ad-a79e-4203-a793-cac6261d74ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3e1a2b83f3d4d88ba5084dfaffd22f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:52:02.476269 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9fa4ad-a79e-4203-a793-cac6261d74ae service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:52:02.476567 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9fa4ad-a79e-4203-a793-cac6261d74ae service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:52:02.476873 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9fa4ad-a79e-4203-a793-cac6261d74ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:52:02.486108 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9fa4ad-a79e-4203-a793-cac6261d74ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:52:02.487561 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9fa4ad-a79e-4203-a793-cac6261d74ae service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:52:02.500075 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1234/2437] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:52:02 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 15:52:02.511606 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1204/2438] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:52:02 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:52:12.497249 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=41 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:52:12.508577 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=51 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:52:36.367683 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729d76ca-72d7-440f-b87b-a2b5cfff483d service nova] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:52:36.369206 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-729d76ca-72d7-440f-b87b-a2b5cfff483d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:52:36.439526 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729d76ca-72d7-440f-b87b-a2b5cfff483d service nova] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:52:36.443115 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a7cf47-20a6-45cc-b75d-25a8864f2306 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:52:36.443794 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a7cf47-20a6-45cc-b75d-25a8864f2306 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:52:36.444107 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a7cf47-20a6-45cc-b75d-25a8864f2306 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:52:36.444399 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a7cf47-20a6-45cc-b75d-25a8864f2306 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:52:36.446275 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-47a7cf47-20a6-45cc-b75d-25a8864f2306 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:52:36.455770 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a7cf47-20a6-45cc-b75d-25a8864f2306 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:52:36.457206 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a7cf47-20a6-45cc-b75d-25a8864f2306 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:52:36.465336 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1235/2439] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:52:36 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 15:52:46.462285 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=31 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:54:02.554100 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9fa4ad-a79e-4203-a793-cac6261d74ae service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:54:02.555801 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-4d9fa4ad-a79e-4203-a793-cac6261d74ae service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:54:02.612997 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9fa4ad-a79e-4203-a793-cac6261d74ae service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:54:02.616875 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8014160e-07f0-4379-b344-4ce12327e2b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3e1a2b83f3d4d88ba5084dfaffd22f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:54:02.617521 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8014160e-07f0-4379-b344-4ce12327e2b0 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:54:02.617811 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8014160e-07f0-4379-b344-4ce12327e2b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:54:02.618092 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8014160e-07f0-4379-b344-4ce12327e2b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:54:02.620038 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-8014160e-07f0-4379-b344-4ce12327e2b0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:54:02.634542 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8014160e-07f0-4379-b344-4ce12327e2b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:54:02.636006 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8014160e-07f0-4379-b344-4ce12327e2b0 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:54:02.647950 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1205/2440] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:54:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 15:54:02.655056 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a7cf47-20a6-45cc-b75d-25a8864f2306 service neutron] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:54:02.666532 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a7cf47-20a6-45cc-b75d-25a8864f2306 service neutron] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:54:02.670173 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87725843-279a-4b4c-8b1d-79bd6cbeb64e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3e1a2b83f3d4d88ba5084dfaffd22f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:54:02.670869 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87725843-279a-4b4c-8b1d-79bd6cbeb64e service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:54:02.671192 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87725843-279a-4b4c-8b1d-79bd6cbeb64e service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:54:02.671476 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87725843-279a-4b4c-8b1d-79bd6cbeb64e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:54:02.673596 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-87725843-279a-4b4c-8b1d-79bd6cbeb64e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:54:02.749082 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87725843-279a-4b4c-8b1d-79bd6cbeb64e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:54:02.750534 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87725843-279a-4b4c-8b1d-79bd6cbeb64e service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:54:02.760408 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1236/2441] 10.222.0.99 () {60 vars in 1396 bytes} [Thu Jul 14 15:54:02 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3260 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:54:02.809764 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8014160e-07f0-4379-b344-4ce12327e2b0 service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:54:02.818723 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8014160e-07f0-4379-b344-4ce12327e2b0 service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:54:02.822745 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68fac92-8000-4188-8635-6efa0dc526af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3e1a2b83f3d4d88ba5084dfaffd22f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:54:02.823582 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68fac92-8000-4188-8635-6efa0dc526af service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:54:02.823906 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68fac92-8000-4188-8635-6efa0dc526af service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:54:02.824186 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68fac92-8000-4188-8635-6efa0dc526af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:54:02.826115 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f68fac92-8000-4188-8635-6efa0dc526af service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:54:02.899580 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68fac92-8000-4188-8635-6efa0dc526af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:54:02.901155 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68fac92-8000-4188-8635-6efa0dc526af service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:54:02.910535 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1206/2442] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:54:02 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:54:02.959050 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87725843-279a-4b4c-8b1d-79bd6cbeb64e service cinder] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:54:02.968120 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87725843-279a-4b4c-8b1d-79bd6cbeb64e service cinder] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:54:02.971736 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee97b5f-ed62-49f8-aa52-cb0aeb4d7946 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3e1a2b83f3d4d88ba5084dfaffd22f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:54:02.972380 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee97b5f-ed62-49f8-aa52-cb0aeb4d7946 service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:54:02.972665 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee97b5f-ed62-49f8-aa52-cb0aeb4d7946 service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:54:02.972945 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee97b5f-ed62-49f8-aa52-cb0aeb4d7946 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:54:02.974802 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-6ee97b5f-ed62-49f8-aa52-cb0aeb4d7946 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:54:03.049191 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee97b5f-ed62-49f8-aa52-cb0aeb4d7946 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:54:03.050638 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee97b5f-ed62-49f8-aa52-cb0aeb4d7946 service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:54:03.060192 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1237/2443] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:54:02 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:54:03.086845 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68fac92-8000-4188-8635-6efa0dc526af service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:54:03.097275 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68fac92-8000-4188-8635-6efa0dc526af service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:54:03.101404 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5548a7e1-a1a5-43cc-8f1f-8122c6b6a5be service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3e1a2b83f3d4d88ba5084dfaffd22f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:54:03.102063 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5548a7e1-a1a5-43cc-8f1f-8122c6b6a5be service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:54:03.102350 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5548a7e1-a1a5-43cc-8f1f-8122c6b6a5be service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:54:03.102631 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5548a7e1-a1a5-43cc-8f1f-8122c6b6a5be service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:54:03.104661 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-5548a7e1-a1a5-43cc-8f1f-8122c6b6a5be service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:54:03.121340 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5548a7e1-a1a5-43cc-8f1f-8122c6b6a5be service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:54:03.122771 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5548a7e1-a1a5-43cc-8f1f-8122c6b6a5be service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:54:03.134465 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1207/2444] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:54:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:54:13.056663 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=66 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:54:13.128924 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=65 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:54:37.732494 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee97b5f-ed62-49f8-aa52-cb0aeb4d7946 service cinder] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:54:37.734059 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-6ee97b5f-ed62-49f8-aa52-cb0aeb4d7946 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:54:37.856272 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee97b5f-ed62-49f8-aa52-cb0aeb4d7946 service cinder] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:54:37.862344 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311d8d3d-9838-469c-8359-e6c334e42e6b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc157c485f794e968a60d8405c9310ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:54:37.863021 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311d8d3d-9838-469c-8359-e6c334e42e6b service placement] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:54:37.863308 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311d8d3d-9838-469c-8359-e6c334e42e6b service placement] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:54:37.863618 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311d8d3d-9838-469c-8359-e6c334e42e6b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:54:37.865502 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-311d8d3d-9838-469c-8359-e6c334e42e6b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:54:37.877770 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311d8d3d-9838-469c-8359-e6c334e42e6b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:54:37.879346 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311d8d3d-9838-469c-8359-e6c334e42e6b service placement] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:54:37.889282 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1238/2445] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:54:37 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:54:47.885594 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=49 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:56:03.435522 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5548a7e1-a1a5-43cc-8f1f-8122c6b6a5be service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:03.509443 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5548a7e1-a1a5-43cc-8f1f-8122c6b6a5be service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:03.515815 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ccc792-278c-4900-92e3-d6ea4ae08d3b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9b1c2622934c5185f3b5f1ea623bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:03.517127 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ccc792-278c-4900-92e3-d6ea4ae08d3b service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:03.517520 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ccc792-278c-4900-92e3-d6ea4ae08d3b service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:03.517841 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ccc792-278c-4900-92e3-d6ea4ae08d3b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:03.519886 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e6ccc792-278c-4900-92e3-d6ea4ae08d3b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:56:03.589652 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ccc792-278c-4900-92e3-d6ea4ae08d3b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:03.591243 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ccc792-278c-4900-92e3-d6ea4ae08d3b service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:03.601071 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1208/2446] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:56:03 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:56:13.596950 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=57 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:57:41.353747 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311d8d3d-9838-469c-8359-e6c334e42e6b service placement] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:57:41.382540 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311d8d3d-9838-469c-8359-e6c334e42e6b service placement] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:57:41.387020 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a39fe9-0cb3-461e-ba4d-03da4607bd3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:57:41.387706 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a39fe9-0cb3-461e-ba4d-03da4607bd3a service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:57:41.388000 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a39fe9-0cb3-461e-ba4d-03da4607bd3a service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:57:41.388282 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a39fe9-0cb3-461e-ba4d-03da4607bd3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:57:41.395734 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a39fe9-0cb3-461e-ba4d-03da4607bd3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:57:41.397205 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a39fe9-0cb3-461e-ba4d-03da4607bd3a service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:57:41.405289 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1239/2447] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:57:41 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 15:57:51.401764 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=15 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:58:03.520285 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ccc792-278c-4900-92e3-d6ea4ae08d3b service nova] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:58:03.520285 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e6ccc792-278c-4900-92e3-d6ea4ae08d3b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:58:03.547528 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ccc792-278c-4900-92e3-d6ea4ae08d3b service nova] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:58:03.553348 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbf1709-8746-4d3d-be15-f2af57af7839 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9b1c2622934c5185f3b5f1ea623bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:58:03.554003 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbf1709-8746-4d3d-be15-f2af57af7839 service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:58:03.554296 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbf1709-8746-4d3d-be15-f2af57af7839 service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:58:03.554584 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbf1709-8746-4d3d-be15-f2af57af7839 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:58:03.557154 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-6dbf1709-8746-4d3d-be15-f2af57af7839 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:58:03.632879 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a39fe9-0cb3-461e-ba4d-03da4607bd3a service neutron] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:58:03.643175 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbf1709-8746-4d3d-be15-f2af57af7839 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:58:03.644842 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbf1709-8746-4d3d-be15-f2af57af7839 service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:58:03.654440 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a39fe9-0cb3-461e-ba4d-03da4607bd3a service neutron] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:58:03.654620 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1209/2448] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:58:03 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:58:03.662364 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4388bd2-c12a-4bf3-b1f6-a59d15b8d54f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3e1a2b83f3d4d88ba5084dfaffd22f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:58:03.663035 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4388bd2-c12a-4bf3-b1f6-a59d15b8d54f service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:58:03.663320 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4388bd2-c12a-4bf3-b1f6-a59d15b8d54f service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:58:03.663628 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4388bd2-c12a-4bf3-b1f6-a59d15b8d54f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:58:03.675908 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4388bd2-c12a-4bf3-b1f6-a59d15b8d54f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:58:03.677335 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4388bd2-c12a-4bf3-b1f6-a59d15b8d54f service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:58:03.687255 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1240/2449] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:58:03 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:58:03.863114 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbf1709-8746-4d3d-be15-f2af57af7839 service nova] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:58:03.874818 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbf1709-8746-4d3d-be15-f2af57af7839 service nova] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:58:03.881033 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2830c4d-1100-44da-890c-2966d70a8af0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9b1c2622934c5185f3b5f1ea623bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:58:03.881770 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2830c4d-1100-44da-890c-2966d70a8af0 service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:58:03.882060 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2830c4d-1100-44da-890c-2966d70a8af0 service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:58:03.882347 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2830c4d-1100-44da-890c-2966d70a8af0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:58:03.884353 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a2830c4d-1100-44da-890c-2966d70a8af0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:58:03.955340 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2830c4d-1100-44da-890c-2966d70a8af0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:58:03.957012 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2830c4d-1100-44da-890c-2966d70a8af0 service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:58:03.966227 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1210/2450] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:58:03 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:58:13.682809 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=10 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:58:13.962069 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=71 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:59:44.733631 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4388bd2-c12a-4bf3-b1f6-a59d15b8d54f service cinder] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:59:44.745606 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4388bd2-c12a-4bf3-b1f6-a59d15b8d54f service cinder] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:59:44.751132 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f44e4e3-79dc-4a3a-a35a-2e6846743b68 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc157c485f794e968a60d8405c9310ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:59:44.751810 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f44e4e3-79dc-4a3a-a35a-2e6846743b68 service placement] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:59:44.752100 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f44e4e3-79dc-4a3a-a35a-2e6846743b68 service placement] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:59:44.752380 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f44e4e3-79dc-4a3a-a35a-2e6846743b68 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:59:44.760988 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f44e4e3-79dc-4a3a-a35a-2e6846743b68 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:59:44.763094 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f44e4e3-79dc-4a3a-a35a-2e6846743b68 service placement] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:59:44.773082 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1241/2451] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 15:59:44 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:59:54.768835 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=6 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:00:03.723082 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2830c4d-1100-44da-890c-2966d70a8af0 service nova] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:00:03.734336 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2830c4d-1100-44da-890c-2966d70a8af0 service nova] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:00:03.738371 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f420b5-e1a0-45fc-a5f9-fe9205d79b55 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3e1a2b83f3d4d88ba5084dfaffd22f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:00:03.739034 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f420b5-e1a0-45fc-a5f9-fe9205d79b55 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:00:03.739389 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f420b5-e1a0-45fc-a5f9-fe9205d79b55 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:00:03.739690 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f420b5-e1a0-45fc-a5f9-fe9205d79b55 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:00:03.752536 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f420b5-e1a0-45fc-a5f9-fe9205d79b55 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:00:03.754002 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f420b5-e1a0-45fc-a5f9-fe9205d79b55 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:00:03.766794 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1211/2452] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:00:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:00:03.773948 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f44e4e3-79dc-4a3a-a35a-2e6846743b68 service placement] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:00:03.784562 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f44e4e3-79dc-4a3a-a35a-2e6846743b68 service placement] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:00:03.789621 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3351563b-7841-4c7c-aca9-60c3798e56ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3e1a2b83f3d4d88ba5084dfaffd22f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:00:03.790274 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3351563b-7841-4c7c-aca9-60c3798e56ea service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:00:03.790561 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3351563b-7841-4c7c-aca9-60c3798e56ea service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:00:03.790845 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3351563b-7841-4c7c-aca9-60c3798e56ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:00:03.800235 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3351563b-7841-4c7c-aca9-60c3798e56ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:00:03.801713 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3351563b-7841-4c7c-aca9-60c3798e56ea service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:00:03.832427 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1242/2453] 10.222.0.99 () {60 vars in 1396 bytes} [Thu Jul 14 16:00:03 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3260 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:00:03.880210 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f420b5-e1a0-45fc-a5f9-fe9205d79b55 service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:00:03.888739 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f420b5-e1a0-45fc-a5f9-fe9205d79b55 service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:00:03.892919 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b54dd40-87ba-48c0-a858-d39c1d073284 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3e1a2b83f3d4d88ba5084dfaffd22f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:00:03.893744 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b54dd40-87ba-48c0-a858-d39c1d073284 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:00:03.894090 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b54dd40-87ba-48c0-a858-d39c1d073284 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:00:03.894309 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b54dd40-87ba-48c0-a858-d39c1d073284 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:00:03.905802 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b54dd40-87ba-48c0-a858-d39c1d073284 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:00:03.907282 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b54dd40-87ba-48c0-a858-d39c1d073284 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:00:03.920010 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1212/2454] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:00:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:00:04.476145 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3351563b-7841-4c7c-aca9-60c3798e56ea service cinder] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:00:04.488064 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3351563b-7841-4c7c-aca9-60c3798e56ea service cinder] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:00:04.493541 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816d5cee-f06c-4d54-8f61-9c1e4086a95c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9b1c2622934c5185f3b5f1ea623bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:00:04.494267 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816d5cee-f06c-4d54-8f61-9c1e4086a95c service nova] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:00:04.494625 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816d5cee-f06c-4d54-8f61-9c1e4086a95c service nova] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:00:04.495008 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816d5cee-f06c-4d54-8f61-9c1e4086a95c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:00:04.504085 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816d5cee-f06c-4d54-8f61-9c1e4086a95c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:00:04.505796 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816d5cee-f06c-4d54-8f61-9c1e4086a95c service nova] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:00:04.537835 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1243/2455] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:00:04 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:00:04.736508 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b54dd40-87ba-48c0-a858-d39c1d073284 service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:00:04.748029 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b54dd40-87ba-48c0-a858-d39c1d073284 service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:00:04.754765 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0010666-8e9f-4ae7-bca5-28fdcbd756ca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9b1c2622934c5185f3b5f1ea623bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:00:04.755424 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0010666-8e9f-4ae7-bca5-28fdcbd756ca service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:00:04.755751 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0010666-8e9f-4ae7-bca5-28fdcbd756ca service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:00:04.756039 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0010666-8e9f-4ae7-bca5-28fdcbd756ca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:00:04.758047 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e0010666-8e9f-4ae7-bca5-28fdcbd756ca service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:00:04.771624 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0010666-8e9f-4ae7-bca5-28fdcbd756ca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:00:04.773255 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0010666-8e9f-4ae7-bca5-28fdcbd756ca service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:00:04.785532 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1213/2456] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:00:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:00:05.191111 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816d5cee-f06c-4d54-8f61-9c1e4086a95c service nova] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:00:05.202802 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816d5cee-f06c-4d54-8f61-9c1e4086a95c service nova] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:00:05.208621 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f58ee22-6a09-437c-a9f0-11f85dcc0467 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9b1c2622934c5185f3b5f1ea623bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:00:05.209287 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f58ee22-6a09-437c-a9f0-11f85dcc0467 service nova] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:00:05.209578 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f58ee22-6a09-437c-a9f0-11f85dcc0467 service nova] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:00:05.209863 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f58ee22-6a09-437c-a9f0-11f85dcc0467 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:00:05.219516 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f58ee22-6a09-437c-a9f0-11f85dcc0467 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:00:05.221189 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f58ee22-6a09-437c-a9f0-11f85dcc0467 service nova] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:00:05.256001 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1244/2457] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:00:05 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:00:14.780499 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=18 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:00:15.248227 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=39 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:02:06.720567 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0010666-8e9f-4ae7-bca5-28fdcbd756ca service nova] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:02:06.729933 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0010666-8e9f-4ae7-bca5-28fdcbd756ca service nova] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:02:06.733540 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f78d243-40cc-43fa-822a-51260db1d25d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3e1a2b83f3d4d88ba5084dfaffd22f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:02:06.734179 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f78d243-40cc-43fa-822a-51260db1d25d service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:02:06.734472 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f78d243-40cc-43fa-822a-51260db1d25d service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:02:06.734756 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f78d243-40cc-43fa-822a-51260db1d25d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:02:06.744076 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f78d243-40cc-43fa-822a-51260db1d25d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:02:06.745499 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f78d243-40cc-43fa-822a-51260db1d25d service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:02:06.773013 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1214/2458] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:02:06 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:02:16.770569 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=13 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:03:03.410317 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f58ee22-6a09-437c-a9f0-11f85dcc0467 service nova] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:03.412306 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-9f58ee22-6a09-437c-a9f0-11f85dcc0467 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:03.485927 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f58ee22-6a09-437c-a9f0-11f85dcc0467 service nova] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:03.491773 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab232bf4-05c7-4af7-a3e5-204fadd30bdc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9b1c2622934c5185f3b5f1ea623bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:03.492431 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab232bf4-05c7-4af7-a3e5-204fadd30bdc service nova] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:03.492719 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab232bf4-05c7-4af7-a3e5-204fadd30bdc service nova] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:03.492998 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab232bf4-05c7-4af7-a3e5-204fadd30bdc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:03.501046 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab232bf4-05c7-4af7-a3e5-204fadd30bdc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:03.502643 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab232bf4-05c7-4af7-a3e5-204fadd30bdc service nova] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:03.534991 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1245/2459] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:03:03 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:03:13.532009 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=41 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:03:48.365341 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f78d243-40cc-43fa-822a-51260db1d25d service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:48.366888 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-6f78d243-40cc-43fa-822a-51260db1d25d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:48.435210 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f78d243-40cc-43fa-822a-51260db1d25d service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:48.438916 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e09678b-730e-4115-89a8-b2b61384fb4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:48.439569 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e09678b-730e-4115-89a8-b2b61384fb4e service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:48.439895 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e09678b-730e-4115-89a8-b2b61384fb4e service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:48.440181 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e09678b-730e-4115-89a8-b2b61384fb4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:48.442278 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-3e09678b-730e-4115-89a8-b2b61384fb4e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:48.452607 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e09678b-730e-4115-89a8-b2b61384fb4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:48.454039 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e09678b-730e-4115-89a8-b2b61384fb4e service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:48.462901 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1215/2460] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:03:48 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:03:58.459013 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=31 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:04:07.101657 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab232bf4-05c7-4af7-a3e5-204fadd30bdc service nova] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:07.103274 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-ab232bf4-05c7-4af7-a3e5-204fadd30bdc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:04:07.177409 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab232bf4-05c7-4af7-a3e5-204fadd30bdc service nova] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:07.181360 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff21522-6f28-4452-9f53-c59d270e886c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3e1a2b83f3d4d88ba5084dfaffd22f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:07.182046 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff21522-6f28-4452-9f53-c59d270e886c service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:07.182338 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff21522-6f28-4452-9f53-c59d270e886c service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:07.182621 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff21522-6f28-4452-9f53-c59d270e886c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:07.192221 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff21522-6f28-4452-9f53-c59d270e886c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:07.194250 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff21522-6f28-4452-9f53-c59d270e886c service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:07.224994 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1246/2461] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:04:07 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:04:17.221938 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=38 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:04:49.282211 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e09678b-730e-4115-89a8-b2b61384fb4e service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:49.283911 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-3e09678b-730e-4115-89a8-b2b61384fb4e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:04:49.397776 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e09678b-730e-4115-89a8-b2b61384fb4e service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:49.403227 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804bb3bb-6796-4f4b-be32-511b3f17a206 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc157c485f794e968a60d8405c9310ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:49.403923 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804bb3bb-6796-4f4b-be32-511b3f17a206 service placement] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:49.404209 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804bb3bb-6796-4f4b-be32-511b3f17a206 service placement] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:49.404500 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804bb3bb-6796-4f4b-be32-511b3f17a206 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:49.406577 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-804bb3bb-6796-4f4b-be32-511b3f17a206 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:04:49.420682 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804bb3bb-6796-4f4b-be32-511b3f17a206 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:49.422386 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804bb3bb-6796-4f4b-be32-511b3f17a206 service placement] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:49.433928 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1216/2462] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:04:49 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:04:59.429413 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=45 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:05:03.258576 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff21522-6f28-4452-9f53-c59d270e886c service cinder] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:05:03.267989 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff21522-6f28-4452-9f53-c59d270e886c service cinder] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:05:03.272514 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700eae4f-9e3a-414b-b882-0ce652737ca8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:05:03.273155 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700eae4f-9e3a-414b-b882-0ce652737ca8 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:05:03.273449 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700eae4f-9e3a-414b-b882-0ce652737ca8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:05:03.273729 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700eae4f-9e3a-414b-b882-0ce652737ca8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:05:03.275628 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-700eae4f-9e3a-414b-b882-0ce652737ca8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:05:03.294203 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700eae4f-9e3a-414b-b882-0ce652737ca8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:05:03.295673 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700eae4f-9e3a-414b-b882-0ce652737ca8 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:05:03.306861 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1247/2463] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:05:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:05:03.726061 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804bb3bb-6796-4f4b-be32-511b3f17a206 service placement] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:05:03.727761 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-804bb3bb-6796-4f4b-be32-511b3f17a206 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:05:03.739453 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804bb3bb-6796-4f4b-be32-511b3f17a206 service placement] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:05:03.743067 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc00995-96e5-4468-90c2-02ed0ce40bf8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:05:03.743755 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc00995-96e5-4468-90c2-02ed0ce40bf8 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:05:03.744060 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc00995-96e5-4468-90c2-02ed0ce40bf8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:05:03.744330 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc00995-96e5-4468-90c2-02ed0ce40bf8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:05:03.746208 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-6dc00995-96e5-4468-90c2-02ed0ce40bf8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:05:03.760058 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc00995-96e5-4468-90c2-02ed0ce40bf8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:05:03.761498 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc00995-96e5-4468-90c2-02ed0ce40bf8 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:05:03.772960 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1217/2464] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:05:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:05:13.302105 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=6 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:05:13.767534 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=6 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:06:06.767047 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700eae4f-9e3a-414b-b882-0ce652737ca8 service neutron] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:06:06.780066 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700eae4f-9e3a-414b-b882-0ce652737ca8 service neutron] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:06:06.786441 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc633073-2320-4982-89f7-1425cc4b9c6b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:06:06.788805 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc633073-2320-4982-89f7-1425cc4b9c6b service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:06:06.789095 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc633073-2320-4982-89f7-1425cc4b9c6b service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:06:06.789378 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc633073-2320-4982-89f7-1425cc4b9c6b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:06:06.791272 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-fc633073-2320-4982-89f7-1425cc4b9c6b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:06:06.913810 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc633073-2320-4982-89f7-1425cc4b9c6b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:06:06.915248 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc633073-2320-4982-89f7-1425cc4b9c6b service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:06:06.938214 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1248/2465] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:06:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:06:07.594496 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc00995-96e5-4468-90c2-02ed0ce40bf8 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:06:07.604584 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc00995-96e5-4468-90c2-02ed0ce40bf8 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:06:07.608174 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a60d5a1-bcab-4bcf-a76c-afec9a242da4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3e1a2b83f3d4d88ba5084dfaffd22f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:06:07.608847 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a60d5a1-bcab-4bcf-a76c-afec9a242da4 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:06:07.609128 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a60d5a1-bcab-4bcf-a76c-afec9a242da4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:06:07.609406 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a60d5a1-bcab-4bcf-a76c-afec9a242da4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:06:07.611293 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-9a60d5a1-bcab-4bcf-a76c-afec9a242da4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:06:07.691337 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a60d5a1-bcab-4bcf-a76c-afec9a242da4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:06:07.692976 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a60d5a1-bcab-4bcf-a76c-afec9a242da4 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:06:07.703754 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1218/2466] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:06:07 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:06:07.761898 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc633073-2320-4982-89f7-1425cc4b9c6b service neutron] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:06:07.772898 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc633073-2320-4982-89f7-1425cc4b9c6b service neutron] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:06:07.776638 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38238490-e7dd-408c-893b-8fba5ee28166 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3e1a2b83f3d4d88ba5084dfaffd22f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:06:07.777312 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38238490-e7dd-408c-893b-8fba5ee28166 service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:06:07.777599 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38238490-e7dd-408c-893b-8fba5ee28166 service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:06:07.777880 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38238490-e7dd-408c-893b-8fba5ee28166 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:06:07.779892 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-38238490-e7dd-408c-893b-8fba5ee28166 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:06:07.857476 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38238490-e7dd-408c-893b-8fba5ee28166 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:06:07.858853 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38238490-e7dd-408c-893b-8fba5ee28166 service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:06:07.868240 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1249/2467] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:06:07 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:06:07.963810 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a60d5a1-bcab-4bcf-a76c-afec9a242da4 service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:06:07.972737 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a60d5a1-bcab-4bcf-a76c-afec9a242da4 service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:06:07.976459 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3852eb-039d-432d-a8ff-4497e27070b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3e1a2b83f3d4d88ba5084dfaffd22f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:06:07.977114 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3852eb-039d-432d-a8ff-4497e27070b0 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:06:07.977551 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3852eb-039d-432d-a8ff-4497e27070b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:06:07.977680 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3852eb-039d-432d-a8ff-4497e27070b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:06:07.979584 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-1d3852eb-039d-432d-a8ff-4497e27070b0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:06:08.069815 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3852eb-039d-432d-a8ff-4497e27070b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:06:08.071580 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3852eb-039d-432d-a8ff-4497e27070b0 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:06:08.083080 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1219/2468] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:06:07 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:06:09.076215 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38238490-e7dd-408c-893b-8fba5ee28166 service cinder] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:06:09.094483 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38238490-e7dd-408c-893b-8fba5ee28166 service cinder] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:06:09.100126 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95043f8-3ede-480a-baa4-88dafd0902f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:06:09.101118 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95043f8-3ede-480a-baa4-88dafd0902f2 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:06:09.101617 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95043f8-3ede-480a-baa4-88dafd0902f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:06:09.102059 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95043f8-3ede-480a-baa4-88dafd0902f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:06:09.104694 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-c95043f8-3ede-480a-baa4-88dafd0902f2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:06:09.119216 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95043f8-3ede-480a-baa4-88dafd0902f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:06:09.120983 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95043f8-3ede-480a-baa4-88dafd0902f2 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:06:09.135004 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1250/2469] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:06:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:06:10.944322 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3852eb-039d-432d-a8ff-4497e27070b0 service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:06:10.962969 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3852eb-039d-432d-a8ff-4497e27070b0 service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:06:10.969290 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52e9dd3-ab59-4f8a-adc6-75fa15c84c64 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc157c485f794e968a60d8405c9310ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:06:10.970246 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52e9dd3-ab59-4f8a-adc6-75fa15c84c64 service placement] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:06:10.971473 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52e9dd3-ab59-4f8a-adc6-75fa15c84c64 service placement] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:06:10.971473 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52e9dd3-ab59-4f8a-adc6-75fa15c84c64 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:06:10.973764 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d52e9dd3-ab59-4f8a-adc6-75fa15c84c64 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:06:11.004313 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52e9dd3-ab59-4f8a-adc6-75fa15c84c64 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:06:11.008478 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52e9dd3-ab59-4f8a-adc6-75fa15c84c64 service placement] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:06:11.026133 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1220/2470] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:06:10 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 16:06:13.576085 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95043f8-3ede-480a-baa4-88dafd0902f2 service neutron] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:06:13.589619 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95043f8-3ede-480a-baa4-88dafd0902f2 service neutron] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:06:13.594953 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a679ad4-be4f-47a8-a828-8e3227accdf9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc157c485f794e968a60d8405c9310ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:06:13.596732 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a679ad4-be4f-47a8-a828-8e3227accdf9 service placement] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:06:13.597019 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a679ad4-be4f-47a8-a828-8e3227accdf9 service placement] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:06:13.597321 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a679ad4-be4f-47a8-a828-8e3227accdf9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:06:13.599545 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-2a679ad4-be4f-47a8-a828-8e3227accdf9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:06:13.613105 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a679ad4-be4f-47a8-a828-8e3227accdf9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:06:13.615130 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a679ad4-be4f-47a8-a828-8e3227accdf9 service placement] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:06:13.628532 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1251/2471] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:06:13 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:06:15.502746 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52e9dd3-ab59-4f8a-adc6-75fa15c84c64 service placement] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:06:15.504627 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d52e9dd3-ab59-4f8a-adc6-75fa15c84c64 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:06:15.521698 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52e9dd3-ab59-4f8a-adc6-75fa15c84c64 service placement] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:06:15.528585 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304e7dfd-2ab1-43bc-98fc-d3edd1b2b02f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:06:15.529869 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304e7dfd-2ab1-43bc-98fc-d3edd1b2b02f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:06:15.530217 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304e7dfd-2ab1-43bc-98fc-d3edd1b2b02f admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:06:15.530694 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304e7dfd-2ab1-43bc-98fc-d3edd1b2b02f admin admin] PATH_INFO: `/v3/users/dbeb3476ffee4611896156633c84265c` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:06:15.535198 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304e7dfd-2ab1-43bc-98fc-d3edd1b2b02f admin admin] RBAC: Authorizing `identity:delete_user(user_id=dbeb3476ffee4611896156633c84265c)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:06:15.537673 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304e7dfd-2ab1-43bc-98fc-d3edd1b2b02f admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:06:15.601600 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-304e7dfd-2ab1-43bc-98fc-d3edd1b2b02f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dbeb3476ffee4611896156633c84265c'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:06:15.604830 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:06:15.614242 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-304e7dfd-2ab1-43bc-98fc-d3edd1b2b02f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dbeb3476ffee4611896156633c84265c'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:06:15.630776 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-304e7dfd-2ab1-43bc-98fc-d3edd1b2b02f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dbeb3476ffee4611896156633c84265c'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:06:15.632407 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-304e7dfd-2ab1-43bc-98fc-d3edd1b2b02f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dbeb3476ffee4611896156633c84265c'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:06:15.633279 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-304e7dfd-2ab1-43bc-98fc-d3edd1b2b02f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dbeb3476ffee4611896156633c84265c'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:06:15.654711 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1221/2472] 10.222.0.99 () {58 vars in 1223 bytes} [Thu Jul 14 16:06:15 2022] DELETE /identity/v3/users/dbeb3476ffee4611896156633c84265c => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:06:15.926723 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a679ad4-be4f-47a8-a828-8e3227accdf9 service placement] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:06:15.940994 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a679ad4-be4f-47a8-a828-8e3227accdf9 service placement] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:06:15.950204 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:06:16.072009 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8fab34-4213-4afa-9fcf-4447820df851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:06:16.073046 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8fab34-4213-4afa-9fcf-4447820df851 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:06:16.073582 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8fab34-4213-4afa-9fcf-4447820df851 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:06:16.073874 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8fab34-4213-4afa-9fcf-4447820df851 admin admin] PATH_INFO: `/v3/projects/16f5da37d8ec473dafd80531f7f60c89` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:06:16.075735 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8fab34-4213-4afa-9fcf-4447820df851 admin admin] RBAC: Authorizing `identity:delete_project(project_id=16f5da37d8ec473dafd80531f7f60c89)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:06:16.078982 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8fab34-4213-4afa-9fcf-4447820df851 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:06:16.138702 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-2f8fab34-4213-4afa-9fcf-4447820df851 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16f5da37d8ec473dafd80531f7f60c89', 'request_id': 'req-2f8fab34-4213-4afa-9fcf-4447820df851'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:06:16.146904 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-2f8fab34-4213-4afa-9fcf-4447820df851 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16f5da37d8ec473dafd80531f7f60c89', 'request_id': 'req-2f8fab34-4213-4afa-9fcf-4447820df851'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:06:16.154242 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-2f8fab34-4213-4afa-9fcf-4447820df851 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16f5da37d8ec473dafd80531f7f60c89', 'request_id': 'req-2f8fab34-4213-4afa-9fcf-4447820df851'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:06:16.173045 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-2f8fab34-4213-4afa-9fcf-4447820df851 admin admin] The token cache is being invalidate because project 16f5da37d8ec473dafd80531f7f60c89 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 16:06:16.173684 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-2f8fab34-4213-4afa-9fcf-4447820df851 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:06:16.174726 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-2f8fab34-4213-4afa-9fcf-4447820df851 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:06:16.180055 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1252/2473] 10.222.0.99 () {58 vars in 1232 bytes} [Thu Jul 14 16:06:15 2022] DELETE /identity/v3/projects/16f5da37d8ec473dafd80531f7f60c89 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:06:16.181881 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304e7dfd-2ab1-43bc-98fc-d3edd1b2b02f admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:06:16.183476 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-304e7dfd-2ab1-43bc-98fc-d3edd1b2b02f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:06:16.198702 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304e7dfd-2ab1-43bc-98fc-d3edd1b2b02f admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:06:16.283947 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f957a3-1684-4801-a427-decfa0f6de09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:06:16.284893 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f957a3-1684-4801-a427-decfa0f6de09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:06:16.285183 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f957a3-1684-4801-a427-decfa0f6de09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:06:16.285466 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f957a3-1684-4801-a427-decfa0f6de09 admin admin] PATH_INFO: `/v3/users/b475d2d866b34290a64141613946dc30` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:06:16.301860 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f957a3-1684-4801-a427-decfa0f6de09 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b475d2d866b34290a64141613946dc30)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:06:16.303942 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f957a3-1684-4801-a427-decfa0f6de09 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:06:16.358700 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-25f957a3-1684-4801-a427-decfa0f6de09 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b475d2d866b34290a64141613946dc30'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:06:16.373253 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-25f957a3-1684-4801-a427-decfa0f6de09 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b475d2d866b34290a64141613946dc30'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:06:16.390190 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-25f957a3-1684-4801-a427-decfa0f6de09 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b475d2d866b34290a64141613946dc30'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:06:16.391283 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-25f957a3-1684-4801-a427-decfa0f6de09 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b475d2d866b34290a64141613946dc30'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:06:16.392908 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-25f957a3-1684-4801-a427-decfa0f6de09 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b475d2d866b34290a64141613946dc30'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:06:16.417944 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1222/2474] 10.222.0.99 () {58 vars in 1223 bytes} [Thu Jul 14 16:06:16 2022] DELETE /identity/v3/users/b475d2d866b34290a64141613946dc30 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 16:06:16.685679 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8fab34-4213-4afa-9fcf-4447820df851 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:06:16.687398 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-2f8fab34-4213-4afa-9fcf-4447820df851 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:06:16.702193 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8fab34-4213-4afa-9fcf-4447820df851 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:06:16.768567 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa37044-f1f4-4823-83bf-66bcf45896e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:06:16.769625 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa37044-f1f4-4823-83bf-66bcf45896e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:06:16.769975 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa37044-f1f4-4823-83bf-66bcf45896e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:06:16.770329 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa37044-f1f4-4823-83bf-66bcf45896e1 admin admin] PATH_INFO: `/v3/projects/06dedb413f94489b8009e2db06167a88` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:06:16.782578 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa37044-f1f4-4823-83bf-66bcf45896e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=06dedb413f94489b8009e2db06167a88)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:06:16.785310 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa37044-f1f4-4823-83bf-66bcf45896e1 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:06:16.824957 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:06:16.839848 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-cfa37044-f1f4-4823-83bf-66bcf45896e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06dedb413f94489b8009e2db06167a88', 'request_id': 'req-cfa37044-f1f4-4823-83bf-66bcf45896e1'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:06:16.848588 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-cfa37044-f1f4-4823-83bf-66bcf45896e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06dedb413f94489b8009e2db06167a88', 'request_id': 'req-cfa37044-f1f4-4823-83bf-66bcf45896e1'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:06:16.854790 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-cfa37044-f1f4-4823-83bf-66bcf45896e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06dedb413f94489b8009e2db06167a88', 'request_id': 'req-cfa37044-f1f4-4823-83bf-66bcf45896e1'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:06:16.870659 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-cfa37044-f1f4-4823-83bf-66bcf45896e1 admin admin] The token cache is being invalidate because project 06dedb413f94489b8009e2db06167a88 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 16:06:16.870955 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-cfa37044-f1f4-4823-83bf-66bcf45896e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:06:16.871641 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-cfa37044-f1f4-4823-83bf-66bcf45896e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:06:16.873280 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1253/2475] 10.222.0.99 () {58 vars in 1232 bytes} [Thu Jul 14 16:06:16 2022] DELETE /identity/v3/projects/06dedb413f94489b8009e2db06167a88 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:06:17.863887 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f957a3-1684-4801-a427-decfa0f6de09 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:06:17.864694 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-25f957a3-1684-4801-a427-decfa0f6de09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:06:17.880819 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f957a3-1684-4801-a427-decfa0f6de09 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:06:17.915810 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:06:17.952112 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3d00af-84ec-4483-ae43-e37cf78d61ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:06:17.952994 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3d00af-84ec-4483-ae43-e37cf78d61ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:06:17.953280 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3d00af-84ec-4483-ae43-e37cf78d61ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:06:17.953558 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3d00af-84ec-4483-ae43-e37cf78d61ba admin admin] PATH_INFO: `/v3/users/a0e4900b6ad24eb19fe169d9c8d8c3a7` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:06:17.954796 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3d00af-84ec-4483-ae43-e37cf78d61ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0e4900b6ad24eb19fe169d9c8d8c3a7)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:06:17.957366 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3d00af-84ec-4483-ae43-e37cf78d61ba admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:06:18.018059 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-0e3d00af-84ec-4483-ae43-e37cf78d61ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0e4900b6ad24eb19fe169d9c8d8c3a7'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:06:18.028100 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-0e3d00af-84ec-4483-ae43-e37cf78d61ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0e4900b6ad24eb19fe169d9c8d8c3a7'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:06:18.049797 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-0e3d00af-84ec-4483-ae43-e37cf78d61ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0e4900b6ad24eb19fe169d9c8d8c3a7'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:06:18.051411 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-0e3d00af-84ec-4483-ae43-e37cf78d61ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0e4900b6ad24eb19fe169d9c8d8c3a7'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:06:18.052079 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-0e3d00af-84ec-4483-ae43-e37cf78d61ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0e4900b6ad24eb19fe169d9c8d8c3a7'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:06:18.068094 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1223/2476] 10.222.0.99 () {58 vars in 1223 bytes} [Thu Jul 14 16:06:17 2022] DELETE /identity/v3/users/a0e4900b6ad24eb19fe169d9c8d8c3a7 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:06:18.266217 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa37044-f1f4-4823-83bf-66bcf45896e1 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:06:18.268670 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-cfa37044-f1f4-4823-83bf-66bcf45896e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:06:18.285706 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa37044-f1f4-4823-83bf-66bcf45896e1 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:06:18.347336 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1a9ff5-52f7-4558-b549-29656a17ecf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:06:18.348432 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1a9ff5-52f7-4558-b549-29656a17ecf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:06:18.348797 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1a9ff5-52f7-4558-b549-29656a17ecf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:06:18.349189 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1a9ff5-52f7-4558-b549-29656a17ecf7 admin admin] PATH_INFO: `/v3/projects/599a768f6b9d4134a4b8e05a7e5f5e87` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:06:18.350645 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1a9ff5-52f7-4558-b549-29656a17ecf7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=599a768f6b9d4134a4b8e05a7e5f5e87)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:06:18.353874 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1a9ff5-52f7-4558-b549-29656a17ecf7 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:06:18.421040 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-5d1a9ff5-52f7-4558-b549-29656a17ecf7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '599a768f6b9d4134a4b8e05a7e5f5e87', 'request_id': 'req-5d1a9ff5-52f7-4558-b549-29656a17ecf7'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:06:18.428967 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-5d1a9ff5-52f7-4558-b549-29656a17ecf7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '599a768f6b9d4134a4b8e05a7e5f5e87', 'request_id': 'req-5d1a9ff5-52f7-4558-b549-29656a17ecf7'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:06:18.435553 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-5d1a9ff5-52f7-4558-b549-29656a17ecf7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '599a768f6b9d4134a4b8e05a7e5f5e87', 'request_id': 'req-5d1a9ff5-52f7-4558-b549-29656a17ecf7'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:06:18.449991 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-5d1a9ff5-52f7-4558-b549-29656a17ecf7 admin admin] The token cache is being invalidate because project 599a768f6b9d4134a4b8e05a7e5f5e87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 16:06:18.451145 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-5d1a9ff5-52f7-4558-b549-29656a17ecf7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:06:18.451929 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-5d1a9ff5-52f7-4558-b549-29656a17ecf7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:06:18.454221 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1254/2477] 10.222.0.99 () {58 vars in 1232 bytes} [Thu Jul 14 16:06:18 2022] DELETE /identity/v3/projects/599a768f6b9d4134a4b8e05a7e5f5e87 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:06:18.459384 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3d00af-84ec-4483-ae43-e37cf78d61ba admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:06:18.464828 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0e3d00af-84ec-4483-ae43-e37cf78d61ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:06:18.481302 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3d00af-84ec-4483-ae43-e37cf78d61ba admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:06:18.542804 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9eaa561-6ab1-4933-947f-2f48005a2ab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:06:18.543701 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eaa561-6ab1-4933-947f-2f48005a2ab6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:06:18.543995 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eaa561-6ab1-4933-947f-2f48005a2ab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:06:18.544285 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eaa561-6ab1-4933-947f-2f48005a2ab6 admin admin] PATH_INFO: `/v3/users/99ed6f809fbc4b758e26f55c0e637e67` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:06:18.561089 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9eaa561-6ab1-4933-947f-2f48005a2ab6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=99ed6f809fbc4b758e26f55c0e637e67)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:06:18.563129 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9eaa561-6ab1-4933-947f-2f48005a2ab6 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:06:18.609945 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:06:18.614007 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-f9eaa561-6ab1-4933-947f-2f48005a2ab6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99ed6f809fbc4b758e26f55c0e637e67'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:06:18.626005 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-f9eaa561-6ab1-4933-947f-2f48005a2ab6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99ed6f809fbc4b758e26f55c0e637e67'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:06:18.638854 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-f9eaa561-6ab1-4933-947f-2f48005a2ab6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99ed6f809fbc4b758e26f55c0e637e67'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:06:18.639558 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-f9eaa561-6ab1-4933-947f-2f48005a2ab6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99ed6f809fbc4b758e26f55c0e637e67'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:06:18.640206 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-f9eaa561-6ab1-4933-947f-2f48005a2ab6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99ed6f809fbc4b758e26f55c0e637e67'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:06:18.653413 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1224/2478] 10.222.0.99 () {58 vars in 1223 bytes} [Thu Jul 14 16:06:18 2022] DELETE /identity/v3/users/99ed6f809fbc4b758e26f55c0e637e67 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 16:06:18.845887 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1a9ff5-52f7-4558-b549-29656a17ecf7 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:06:18.848047 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-5d1a9ff5-52f7-4558-b549-29656a17ecf7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:06:18.862156 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1a9ff5-52f7-4558-b549-29656a17ecf7 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:06:18.919041 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:06:18.940742 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff827c6-0cf4-41e6-8e5a-ec886b8a1ea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:06:18.942280 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff827c6-0cf4-41e6-8e5a-ec886b8a1ea1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:06:18.942672 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff827c6-0cf4-41e6-8e5a-ec886b8a1ea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:06:18.942958 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff827c6-0cf4-41e6-8e5a-ec886b8a1ea1 admin admin] PATH_INFO: `/v3/projects/5f1c8e0732c4436ca32e38bd3b352e2a` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:06:18.954228 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff827c6-0cf4-41e6-8e5a-ec886b8a1ea1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f1c8e0732c4436ca32e38bd3b352e2a)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:06:18.957147 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff827c6-0cf4-41e6-8e5a-ec886b8a1ea1 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:06:19.010440 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-dff827c6-0cf4-41e6-8e5a-ec886b8a1ea1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f1c8e0732c4436ca32e38bd3b352e2a', 'request_id': 'req-dff827c6-0cf4-41e6-8e5a-ec886b8a1ea1'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:06:19.018135 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-dff827c6-0cf4-41e6-8e5a-ec886b8a1ea1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f1c8e0732c4436ca32e38bd3b352e2a', 'request_id': 'req-dff827c6-0cf4-41e6-8e5a-ec886b8a1ea1'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:06:19.023883 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-dff827c6-0cf4-41e6-8e5a-ec886b8a1ea1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f1c8e0732c4436ca32e38bd3b352e2a', 'request_id': 'req-dff827c6-0cf4-41e6-8e5a-ec886b8a1ea1'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:06:19.039032 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-dff827c6-0cf4-41e6-8e5a-ec886b8a1ea1 admin admin] The token cache is being invalidate because project 5f1c8e0732c4436ca32e38bd3b352e2a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 16:06:19.039324 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-dff827c6-0cf4-41e6-8e5a-ec886b8a1ea1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:06:19.040006 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-dff827c6-0cf4-41e6-8e5a-ec886b8a1ea1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:06:19.041797 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1255/2479] 10.222.0.99 () {58 vars in 1232 bytes} [Thu Jul 14 16:06:18 2022] DELETE /identity/v3/projects/5f1c8e0732c4436ca32e38bd3b352e2a => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:06:28.649835 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:DELETE=4,keystone:SELECT=13,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:06:29.035117 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=33,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:07:02.388310 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9eaa561-6ab1-4933-947f-2f48005a2ab6 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:07:02.389752 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f9eaa561-6ab1-4933-947f-2f48005a2ab6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:07:02.522755 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9eaa561-6ab1-4933-947f-2f48005a2ab6 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:07:02.601814 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197de80a-98dd-44e7-a33b-fc558b8362a0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9b1c2622934c5185f3b5f1ea623bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:07:02.602562 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197de80a-98dd-44e7-a33b-fc558b8362a0 service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:07:02.603195 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197de80a-98dd-44e7-a33b-fc558b8362a0 service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:07:02.603637 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197de80a-98dd-44e7-a33b-fc558b8362a0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:07:02.605606 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-197de80a-98dd-44e7-a33b-fc558b8362a0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:07:02.627915 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197de80a-98dd-44e7-a33b-fc558b8362a0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:07:02.629554 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197de80a-98dd-44e7-a33b-fc558b8362a0 service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:07:02.643978 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1225/2480] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:07:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:07:05.964177 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff827c6-0cf4-41e6-8e5a-ec886b8a1ea1 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:07:05.965572 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-dff827c6-0cf4-41e6-8e5a-ec886b8a1ea1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:07:05.984240 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff827c6-0cf4-41e6-8e5a-ec886b8a1ea1 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:07:05.992898 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0179fc0-04da-4ee8-ad63-73c6daae6303 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9b1c2622934c5185f3b5f1ea623bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:07:05.993702 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0179fc0-04da-4ee8-ad63-73c6daae6303 service nova] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:07:05.993992 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0179fc0-04da-4ee8-ad63-73c6daae6303 service nova] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:07:05.994270 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0179fc0-04da-4ee8-ad63-73c6daae6303 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:07:05.996420 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-e0179fc0-04da-4ee8-ad63-73c6daae6303 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:07:06.074981 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0179fc0-04da-4ee8-ad63-73c6daae6303 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:07:06.076609 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0179fc0-04da-4ee8-ad63-73c6daae6303 service nova] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:07:06.086148 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1256/2481] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:07:05 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:07:12.644409 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=69 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:07:16.081566 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=33 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:08:51.349760 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197de80a-98dd-44e7-a33b-fc558b8362a0 service nova] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:08:51.351208 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-197de80a-98dd-44e7-a33b-fc558b8362a0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:08:51.429148 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197de80a-98dd-44e7-a33b-fc558b8362a0 service nova] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:08:51.432730 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082844c2-4791-46d7-a2f0-eff0af1389d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:08:51.433377 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082844c2-4791-46d7-a2f0-eff0af1389d2 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:08:51.433661 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082844c2-4791-46d7-a2f0-eff0af1389d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:08:51.434201 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082844c2-4791-46d7-a2f0-eff0af1389d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:08:51.435853 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-082844c2-4791-46d7-a2f0-eff0af1389d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:08:51.445263 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082844c2-4791-46d7-a2f0-eff0af1389d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:08:51.446686 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082844c2-4791-46d7-a2f0-eff0af1389d2 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:08:51.455011 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1226/2482] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:08:51 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:09:01.205019 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0179fc0-04da-4ee8-ad63-73c6daae6303 service nova] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:09:01.206432 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-e0179fc0-04da-4ee8-ad63-73c6daae6303 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:09:01.287257 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0179fc0-04da-4ee8-ad63-73c6daae6303 service nova] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:09:01.291197 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbe3074-b8e5-424e-abbd-5479a48626e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3e1a2b83f3d4d88ba5084dfaffd22f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:09:01.291888 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbe3074-b8e5-424e-abbd-5479a48626e1 service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:09:01.292175 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbe3074-b8e5-424e-abbd-5479a48626e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:09:01.292452 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbe3074-b8e5-424e-abbd-5479a48626e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:09:01.294573 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-6dbe3074-b8e5-424e-abbd-5479a48626e1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:09:01.307912 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application [None req-6dbe3074-b8e5-424e-abbd-5479a48626e1 service cinder] Could not find project: 16f5da37d8ec473dafd80531f7f60c89.: keystone.exception.ProjectNotFound: Could not find project: 16f5da37d8ec473dafd80531f7f60c89. Jul 14 16:09:01.307912 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 16:09:01.307912 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 16:09:01.307912 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 16:09:01.307912 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 16:09:01.307912 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 16:09:01.307912 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 16:09:01.307912 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 16:09:01.307912 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 16:09:01.307912 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 16:09:01.307912 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 16:09:01.307912 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 16:09:01.307912 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 285, in get Jul 14 16:09:01.307912 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:validate_token') Jul 14 16:09:01.307912 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 422, in enforce_call Jul 14 16:09:01.309171 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application subj_token_target_data = cls._extract_subject_token_target_data() Jul 14 16:09:01.309171 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 261, in _extract_subject_token_target_data Jul 14 16:09:01.309171 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application token = PROVIDER_APIS.token_provider_api.validate_token( Jul 14 16:09:01.309171 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 16:09:01.309171 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 16:09:01.309171 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 145, in validate_token Jul 14 16:09:01.309171 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application token = self._validate_token(token_id) Jul 14 16:09:01.309171 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 16:09:01.309171 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 16:09:01.309171 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 16:09:01.309171 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 16:09:01.309171 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 16:09:01.309171 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 16:09:01.309171 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 16:09:01.309171 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 16:09:01.309171 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 16:09:01.309171 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 16:09:01.312128 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 16:09:01.312128 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 16:09:01.312128 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 16:09:01.312128 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 16:09:01.312128 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 179, in _validate_token Jul 14 16:09:01.312128 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application token.mint(token_id, issued_at) Jul 14 16:09:01.312128 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 579, in mint Jul 14 16:09:01.312128 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application self._validate_token_resources() Jul 14 16:09:01.312128 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 464, in _validate_token_resources Jul 14 16:09:01.312128 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application if self.project and not self.project.get('enabled'): Jul 14 16:09:01.312128 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 162, in project Jul 14 16:09:01.312128 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application self.__project = PROVIDERS.resource_api.get_project( Jul 14 16:09:01.312128 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 16:09:01.312128 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 16:09:01.312128 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 16:09:01.312128 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 16:09:01.312128 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 16:09:01.314020 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 16:09:01.314020 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 16:09:01.314020 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application with Lock( Jul 14 16:09:01.314020 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 16:09:01.314020 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._enter() Jul 14 16:09:01.314020 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 16:09:01.314020 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 16:09:01.314020 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 16:09:01.314020 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.creator() Jul 14 16:09:01.314020 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 16:09:01.314020 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application created_value = creator( Jul 14 16:09:01.314020 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 16:09:01.314020 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 16:09:01.314020 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 16:09:01.314020 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 16:09:01.314020 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 16:09:01.314020 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 16:09:01.315569 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 16f5da37d8ec473dafd80531f7f60c89. Jul 14 16:09:01.315569 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: ERROR keystone.server.flask.application Jul 14 16:09:01.315569 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1257/2483] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:09:01 2022] GET /identity/v3/auth/tokens => generated 113 bytes in 105 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Jul 14 16:09:01.451726 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=35 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:09:11.298530 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=33 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:09:53.711133 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082844c2-4791-46d7-a2f0-eff0af1389d2 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:09:53.712712 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-082844c2-4791-46d7-a2f0-eff0af1389d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:09:53.785376 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082844c2-4791-46d7-a2f0-eff0af1389d2 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:09:53.790362 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11406a9-6440-4f8d-9127-a4450e64314a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc157c485f794e968a60d8405c9310ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:09:53.791025 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11406a9-6440-4f8d-9127-a4450e64314a service placement] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:09:53.791336 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11406a9-6440-4f8d-9127-a4450e64314a service placement] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:09:53.791639 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11406a9-6440-4f8d-9127-a4450e64314a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:09:53.793943 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c11406a9-6440-4f8d-9127-a4450e64314a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:09:53.806718 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11406a9-6440-4f8d-9127-a4450e64314a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:09:53.808342 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11406a9-6440-4f8d-9127-a4450e64314a service placement] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:09:53.818742 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1227/2484] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:09:53 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:10:03.571106 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbe3074-b8e5-424e-abbd-5479a48626e1 service cinder] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:10:03.584985 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbe3074-b8e5-424e-abbd-5479a48626e1 service cinder] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:10:03.591496 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba09731-e75e-40a8-8217-6aeb7db23b41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:10:03.592182 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba09731-e75e-40a8-8217-6aeb7db23b41 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:10:03.592470 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba09731-e75e-40a8-8217-6aeb7db23b41 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:10:03.592755 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba09731-e75e-40a8-8217-6aeb7db23b41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:10:03.594588 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-fba09731-e75e-40a8-8217-6aeb7db23b41 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:10:03.629128 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba09731-e75e-40a8-8217-6aeb7db23b41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:10:03.630603 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba09731-e75e-40a8-8217-6aeb7db23b41 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:10:03.645029 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1258/2485] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:10:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:10:03.809418 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11406a9-6440-4f8d-9127-a4450e64314a service placement] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:10:03.810771 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c11406a9-6440-4f8d-9127-a4450e64314a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:10:03.825369 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11406a9-6440-4f8d-9127-a4450e64314a service placement] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:10:03.830558 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d04de16-8150-46a2-9784-08a9c0f27584 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:10:03.831455 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d04de16-8150-46a2-9784-08a9c0f27584 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:10:03.831662 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d04de16-8150-46a2-9784-08a9c0f27584 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:10:03.831887 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d04de16-8150-46a2-9784-08a9c0f27584 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:10:03.833801 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0d04de16-8150-46a2-9784-08a9c0f27584 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:10:03.850828 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d04de16-8150-46a2-9784-08a9c0f27584 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:10:03.852662 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d04de16-8150-46a2-9784-08a9c0f27584 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:10:03.870693 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1228/2486] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:10:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:10:11.082199 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa55f58b-f7aa-41ed-8d2a-6e2630ca15f8 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:10:11.082799 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa55f58b-f7aa-41ed-8d2a-6e2630ca15f8 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:10:11.082799 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa55f58b-f7aa-41ed-8d2a-6e2630ca15f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:10:11.111157 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-aa55f58b-f7aa-41ed-8d2a-6e2630ca15f8 None None] MFA Rules not processed for user `781567471d0f4ca4add4331d405febd4`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:10:11.112450 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-aa55f58b-f7aa-41ed-8d2a-6e2630ca15f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:10:11.122960 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1259/2487] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 16:10:11 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:10:11.524650 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d04de16-8150-46a2-9784-08a9c0f27584 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:10:11.536419 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d04de16-8150-46a2-9784-08a9c0f27584 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:10:11.541397 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6120c7e2-111a-410e-8451-faf91b647fe0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:10:11.542899 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6120c7e2-111a-410e-8451-faf91b647fe0 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:10:11.542899 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6120c7e2-111a-410e-8451-faf91b647fe0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:10:11.543129 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6120c7e2-111a-410e-8451-faf91b647fe0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:10:11.544928 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-6120c7e2-111a-410e-8451-faf91b647fe0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:10:11.559731 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6120c7e2-111a-410e-8451-faf91b647fe0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:10:11.561098 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6120c7e2-111a-410e-8451-faf91b647fe0 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:10:11.570922 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1229/2488] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:10:11 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 16:10:12.188180 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa55f58b-f7aa-41ed-8d2a-6e2630ca15f8 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:10:12.203308 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa55f58b-f7aa-41ed-8d2a-6e2630ca15f8 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:10:12.213173 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73d77ea-3cc8-44e4-bb34-35cf48f461b6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc157c485f794e968a60d8405c9310ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:10:12.214327 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73d77ea-3cc8-44e4-bb34-35cf48f461b6 service placement] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:10:12.214858 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73d77ea-3cc8-44e4-bb34-35cf48f461b6 service placement] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:10:12.215337 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73d77ea-3cc8-44e4-bb34-35cf48f461b6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:10:12.217323 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-c73d77ea-3cc8-44e4-bb34-35cf48f461b6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:10:12.231256 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73d77ea-3cc8-44e4-bb34-35cf48f461b6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:10:12.233163 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73d77ea-3cc8-44e4-bb34-35cf48f461b6 service placement] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:10:12.256520 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1260/2489] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:10:12 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:10:21.567641 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=44 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:10:22.187938 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6120c7e2-111a-410e-8451-faf91b647fe0 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:10:22.190873 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-6120c7e2-111a-410e-8451-faf91b647fe0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:10:22.212349 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6120c7e2-111a-410e-8451-faf91b647fe0 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:10:22.219108 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c09527-f33f-4ab0-9783-3b2a4197fb0d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9b1c2622934c5185f3b5f1ea623bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:10:22.220503 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c09527-f33f-4ab0-9783-3b2a4197fb0d service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:10:22.220786 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c09527-f33f-4ab0-9783-3b2a4197fb0d service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:10:22.221072 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c09527-f33f-4ab0-9783-3b2a4197fb0d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:10:22.222940 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-78c09527-f33f-4ab0-9783-3b2a4197fb0d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:10:22.236744 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c09527-f33f-4ab0-9783-3b2a4197fb0d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:10:22.238502 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c09527-f33f-4ab0-9783-3b2a4197fb0d service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:10:22.250337 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=23 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:10:22.257986 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1230/2490] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:10:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:10:32.245296 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=6 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:11:07.947917 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73d77ea-3cc8-44e4-bb34-35cf48f461b6 service placement] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:11:07.961978 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73d77ea-3cc8-44e4-bb34-35cf48f461b6 service placement] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:11:07.967412 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11499b9a-b824-4dc7-b8b0-68475ab96428 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9b1c2622934c5185f3b5f1ea623bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:11:07.968092 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11499b9a-b824-4dc7-b8b0-68475ab96428 service nova] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:11:07.968383 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11499b9a-b824-4dc7-b8b0-68475ab96428 service nova] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:11:07.968660 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11499b9a-b824-4dc7-b8b0-68475ab96428 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:11:07.970431 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-11499b9a-b824-4dc7-b8b0-68475ab96428 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:11:08.016556 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c09527-f33f-4ab0-9783-3b2a4197fb0d service nova] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:11:08.028445 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c09527-f33f-4ab0-9783-3b2a4197fb0d service nova] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:11:08.034851 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411b04d5-2906-409d-9590-9863ee94e27a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9b1c2622934c5185f3b5f1ea623bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:11:08.035495 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411b04d5-2906-409d-9590-9863ee94e27a service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:11:08.035807 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411b04d5-2906-409d-9590-9863ee94e27a service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:11:08.036084 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411b04d5-2906-409d-9590-9863ee94e27a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:11:08.038108 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-411b04d5-2906-409d-9590-9863ee94e27a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:11:08.043344 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11499b9a-b824-4dc7-b8b0-68475ab96428 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:11:08.045015 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11499b9a-b824-4dc7-b8b0-68475ab96428 service nova] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:11:08.063996 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1261/2491] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:11:07 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:11:08.109933 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411b04d5-2906-409d-9590-9863ee94e27a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:11:08.111649 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411b04d5-2906-409d-9590-9863ee94e27a service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:11:08.132883 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1231/2492] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:11:08 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 16:11:08.455943 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11499b9a-b824-4dc7-b8b0-68475ab96428 service nova] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:11:08.467362 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11499b9a-b824-4dc7-b8b0-68475ab96428 service nova] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:11:08.472694 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531e86ac-2088-4cec-9c76-a1299e706f31 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9b1c2622934c5185f3b5f1ea623bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:11:08.473346 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531e86ac-2088-4cec-9c76-a1299e706f31 service nova] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:11:08.473655 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531e86ac-2088-4cec-9c76-a1299e706f31 service nova] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:11:08.474537 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531e86ac-2088-4cec-9c76-a1299e706f31 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:11:08.476368 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-531e86ac-2088-4cec-9c76-a1299e706f31 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:11:08.552357 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531e86ac-2088-4cec-9c76-a1299e706f31 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:11:08.554162 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531e86ac-2088-4cec-9c76-a1299e706f31 service nova] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:11:08.572990 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1262/2493] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:11:08 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:11:18.129045 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=36 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:11:18.570130 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=72 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:12:10.397981 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411b04d5-2906-409d-9590-9863ee94e27a service nova] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:12:10.409357 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411b04d5-2906-409d-9590-9863ee94e27a service nova] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:12:10.414796 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe3859d-2e96-4a3a-81d4-4adb8385c0a8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9b1c2622934c5185f3b5f1ea623bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:12:10.415444 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe3859d-2e96-4a3a-81d4-4adb8385c0a8 service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:12:10.415780 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe3859d-2e96-4a3a-81d4-4adb8385c0a8 service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:12:10.416059 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe3859d-2e96-4a3a-81d4-4adb8385c0a8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:12:10.424758 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe3859d-2e96-4a3a-81d4-4adb8385c0a8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:12:10.426348 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe3859d-2e96-4a3a-81d4-4adb8385c0a8 service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:12:10.456125 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1232/2494] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:12:10 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:12:13.301731 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531e86ac-2088-4cec-9c76-a1299e706f31 service nova] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:12:13.317421 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531e86ac-2088-4cec-9c76-a1299e706f31 service nova] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:12:13.323263 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed82512-34a4-4b3a-9a0e-2389fa07d477 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9b1c2622934c5185f3b5f1ea623bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:12:13.324054 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed82512-34a4-4b3a-9a0e-2389fa07d477 service nova] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:12:13.324424 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed82512-34a4-4b3a-9a0e-2389fa07d477 service nova] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:12:13.324777 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed82512-34a4-4b3a-9a0e-2389fa07d477 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:12:13.336654 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed82512-34a4-4b3a-9a0e-2389fa07d477 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:12:13.338444 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed82512-34a4-4b3a-9a0e-2389fa07d477 service nova] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:12:13.362008 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1263/2495] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:12:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:12:20.454440 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=13 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:12:23.346590 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=6 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:13:02.853022 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f5c65f-3dca-4b38-b57b-c82cef97adc6 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:02.853578 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f5c65f-3dca-4b38-b57b-c82cef97adc6 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:02.853578 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f5c65f-3dca-4b38-b57b-c82cef97adc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:02.880471 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-f7f5c65f-3dca-4b38-b57b-c82cef97adc6 None None] MFA Rules not processed for user `46f52ad195a34123982d238ea351172d`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:13:02.881782 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f7f5c65f-3dca-4b38-b57b-c82cef97adc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:13:02.889121 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1233/2496] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 16:13:02 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:13:03.225866 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed82512-34a4-4b3a-9a0e-2389fa07d477 service nova] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:13:03.235915 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed82512-34a4-4b3a-9a0e-2389fa07d477 service nova] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:13:03.240364 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4791005-f69e-4de7-b768-f38154f594c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:13:03.241020 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4791005-f69e-4de7-b768-f38154f594c6 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:03.241311 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4791005-f69e-4de7-b768-f38154f594c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:03.241593 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4791005-f69e-4de7-b768-f38154f594c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:03.243500 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-f4791005-f69e-4de7-b768-f38154f594c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:13:03.253526 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4791005-f69e-4de7-b768-f38154f594c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:13:03.254954 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4791005-f69e-4de7-b768-f38154f594c6 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:13:03.263097 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1264/2497] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:13:03 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:13:12.878411 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=6 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:13:13.259732 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=6 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:13:15.248072 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f5c65f-3dca-4b38-b57b-c82cef97adc6 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:13:15.262483 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f5c65f-3dca-4b38-b57b-c82cef97adc6 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:13:15.269183 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c290d7-f9c7-44d2-a5aa-691c7afe0122 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9b1c2622934c5185f3b5f1ea623bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:13:15.269998 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c290d7-f9c7-44d2-a5aa-691c7afe0122 service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:15.270268 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c290d7-f9c7-44d2-a5aa-691c7afe0122 service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:15.270580 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c290d7-f9c7-44d2-a5aa-691c7afe0122 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:15.272598 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c1c290d7-f9c7-44d2-a5aa-691c7afe0122 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:13:15.287194 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c290d7-f9c7-44d2-a5aa-691c7afe0122 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:13:15.289097 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c290d7-f9c7-44d2-a5aa-691c7afe0122 service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:13:15.304773 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1234/2498] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:13:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:13:16.514953 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4791005-f69e-4de7-b768-f38154f594c6 service neutron] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:13:16.525384 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4791005-f69e-4de7-b768-f38154f594c6 service neutron] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:13:16.529139 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2beddc-ce13-4c86-ad06-bab176c00225 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:13:16.529819 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2beddc-ce13-4c86-ad06-bab176c00225 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:16.530171 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2beddc-ce13-4c86-ad06-bab176c00225 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:16.530394 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2beddc-ce13-4c86-ad06-bab176c00225 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:16.532408 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-aa2beddc-ce13-4c86-ad06-bab176c00225 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:13:16.605293 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2beddc-ce13-4c86-ad06-bab176c00225 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:13:16.606784 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2beddc-ce13-4c86-ad06-bab176c00225 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:13:16.617538 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1265/2499] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:13:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 16:13:20.209367 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c290d7-f9c7-44d2-a5aa-691c7afe0122 service nova] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:13:20.225110 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c290d7-f9c7-44d2-a5aa-691c7afe0122 service nova] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:13:20.230351 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15dc65f-b2dc-4475-8e2c-98ad119a8573 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc157c485f794e968a60d8405c9310ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:13:20.231005 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15dc65f-b2dc-4475-8e2c-98ad119a8573 service placement] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:20.231295 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15dc65f-b2dc-4475-8e2c-98ad119a8573 service placement] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:20.231575 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15dc65f-b2dc-4475-8e2c-98ad119a8573 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:20.233410 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b15dc65f-b2dc-4475-8e2c-98ad119a8573 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:13:20.245735 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15dc65f-b2dc-4475-8e2c-98ad119a8573 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:13:20.247546 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15dc65f-b2dc-4475-8e2c-98ad119a8573 service placement] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:13:20.259188 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1235/2500] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:13:20 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:13:20.284097 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2beddc-ce13-4c86-ad06-bab176c00225 service neutron] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:13:20.286428 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-aa2beddc-ce13-4c86-ad06-bab176c00225 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:13:20.305911 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2beddc-ce13-4c86-ad06-bab176c00225 service neutron] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:13:20.313422 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059c05a6-b456-4da8-93aa-c48229c8026e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:13:20.315533 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059c05a6-b456-4da8-93aa-c48229c8026e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:20.319072 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059c05a6-b456-4da8-93aa-c48229c8026e admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:20.319072 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059c05a6-b456-4da8-93aa-c48229c8026e admin admin] PATH_INFO: `/v3/users/d01c301ea20d4acab8dff1148501a975` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:20.319736 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059c05a6-b456-4da8-93aa-c48229c8026e admin admin] RBAC: Authorizing `identity:delete_user(user_id=d01c301ea20d4acab8dff1148501a975)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:13:20.321854 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059c05a6-b456-4da8-93aa-c48229c8026e admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:13:20.383038 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-059c05a6-b456-4da8-93aa-c48229c8026e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd01c301ea20d4acab8dff1148501a975'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:13:20.391773 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-059c05a6-b456-4da8-93aa-c48229c8026e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd01c301ea20d4acab8dff1148501a975'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:13:20.405894 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-059c05a6-b456-4da8-93aa-c48229c8026e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd01c301ea20d4acab8dff1148501a975'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:13:20.406609 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-059c05a6-b456-4da8-93aa-c48229c8026e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd01c301ea20d4acab8dff1148501a975'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:13:20.407206 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-059c05a6-b456-4da8-93aa-c48229c8026e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd01c301ea20d4acab8dff1148501a975'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:13:20.423326 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1266/2501] 10.222.0.99 () {58 vars in 1223 bytes} [Thu Jul 14 16:13:20 2022] DELETE /identity/v3/users/d01c301ea20d4acab8dff1148501a975 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:13:20.664531 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15dc65f-b2dc-4475-8e2c-98ad119a8573 service placement] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:13:20.678329 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15dc65f-b2dc-4475-8e2c-98ad119a8573 service placement] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:13:20.750129 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf2e798-745d-4370-9c59-e3a99cab9ce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:13:20.751020 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf2e798-745d-4370-9c59-e3a99cab9ce1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:20.751307 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf2e798-745d-4370-9c59-e3a99cab9ce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:20.751589 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf2e798-745d-4370-9c59-e3a99cab9ce1 admin admin] PATH_INFO: `/v3/projects/2a542b3ba16943a4ab7d2624d961b4a5` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:20.752867 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf2e798-745d-4370-9c59-e3a99cab9ce1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a542b3ba16943a4ab7d2624d961b4a5)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:13:20.755174 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf2e798-745d-4370-9c59-e3a99cab9ce1 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:13:20.817417 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-7bf2e798-745d-4370-9c59-e3a99cab9ce1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a542b3ba16943a4ab7d2624d961b4a5', 'request_id': 'req-7bf2e798-745d-4370-9c59-e3a99cab9ce1'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:13:20.824233 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-7bf2e798-745d-4370-9c59-e3a99cab9ce1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a542b3ba16943a4ab7d2624d961b4a5', 'request_id': 'req-7bf2e798-745d-4370-9c59-e3a99cab9ce1'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:13:20.830300 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-7bf2e798-745d-4370-9c59-e3a99cab9ce1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a542b3ba16943a4ab7d2624d961b4a5', 'request_id': 'req-7bf2e798-745d-4370-9c59-e3a99cab9ce1'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:13:20.844945 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-7bf2e798-745d-4370-9c59-e3a99cab9ce1 admin admin] The token cache is being invalidate because project 2a542b3ba16943a4ab7d2624d961b4a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 16:13:20.845226 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-7bf2e798-745d-4370-9c59-e3a99cab9ce1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:13:20.845878 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-7bf2e798-745d-4370-9c59-e3a99cab9ce1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:13:20.851552 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1236/2502] 10.222.0.99 () {58 vars in 1232 bytes} [Thu Jul 14 16:13:20 2022] DELETE /identity/v3/projects/2a542b3ba16943a4ab7d2624d961b4a5 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 16:13:20.854291 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ead5380-7b05-4ac3-be7a-b085fc735aae None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:20.854600 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ead5380-7b05-4ac3-be7a-b085fc735aae None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:20.854886 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ead5380-7b05-4ac3-be7a-b085fc735aae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:20.925644 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-7ead5380-7b05-4ac3-be7a-b085fc735aae None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:13:20.927145 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-7ead5380-7b05-4ac3-be7a-b085fc735aae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:13:20.963789 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:13:21.012227 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1267/2503] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 16:13:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 16:13:21.022361 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5496d0e4-35a2-4aaa-a18c-c37b7c140425 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:21.022846 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5496d0e4-35a2-4aaa-a18c-c37b7c140425 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:21.023203 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5496d0e4-35a2-4aaa-a18c-c37b7c140425 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:21.044351 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-5496d0e4-35a2-4aaa-a18c-c37b7c140425 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:13:21.045864 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-5496d0e4-35a2-4aaa-a18c-c37b7c140425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:13:21.055036 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1237/2504] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 16:13:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:13:21.059546 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ead5380-7b05-4ac3-be7a-b085fc735aae None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:13:21.073469 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ead5380-7b05-4ac3-be7a-b085fc735aae None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:13:21.078352 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe6b351-2cc9-4987-828c-b3587a231790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:13:21.079026 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe6b351-2cc9-4987-828c-b3587a231790 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:21.079311 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe6b351-2cc9-4987-828c-b3587a231790 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:21.079588 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe6b351-2cc9-4987-828c-b3587a231790 admin admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:21.080376 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe6b351-2cc9-4987-828c-b3587a231790 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 16:13:21.080682 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe6b351-2cc9-4987-828c-b3587a231790 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:13:21.082461 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe6b351-2cc9-4987-828c-b3587a231790 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:13:21.095974 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1268/2505] 10.222.0.99 () {58 vars in 1152 bytes} [Thu Jul 14 16:13:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 16:13:21.101102 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5496d0e4-35a2-4aaa-a18c-c37b7c140425 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:13:21.111862 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5496d0e4-35a2-4aaa-a18c-c37b7c140425 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:13:21.117966 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f265b88c-d592-4c46-a039-fbb7562e1dcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:13:21.118987 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f265b88c-d592-4c46-a039-fbb7562e1dcf admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:21.119305 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f265b88c-d592-4c46-a039-fbb7562e1dcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:21.119697 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f265b88c-d592-4c46-a039-fbb7562e1dcf admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:21.120859 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f265b88c-d592-4c46-a039-fbb7562e1dcf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:13:21.123013 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f265b88c-d592-4c46-a039-fbb7562e1dcf admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:13:21.180803 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1238/2506] 10.222.0.99 () {60 vars in 1152 bytes} [Thu Jul 14 16:13:21 2022] POST /identity/v3/projects => generated 403 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 16:13:21.185591 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe6b351-2cc9-4987-828c-b3587a231790 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:13:21.196564 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe6b351-2cc9-4987-828c-b3587a231790 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:13:21.258086 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52839f9-a0b5-457e-b45d-6a79218303a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:13:21.259100 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52839f9-a0b5-457e-b45d-6a79218303a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:21.259391 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52839f9-a0b5-457e-b45d-6a79218303a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:21.259709 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52839f9-a0b5-457e-b45d-6a79218303a0 admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:21.260409 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52839f9-a0b5-457e-b45d-6a79218303a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:13:21.262373 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52839f9-a0b5-457e-b45d-6a79218303a0 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:13:21.284532 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1269/2507] 10.222.0.99 () {60 vars in 1143 bytes} [Thu Jul 14 16:13:21 2022] POST /identity/v3/users => generated 302 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 16:13:21.289522 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f265b88c-d592-4c46-a039-fbb7562e1dcf admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:13:21.301996 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f265b88c-d592-4c46-a039-fbb7562e1dcf admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:13:21.307567 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be934bef-2220-45d2-b53d-21e360f10b0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:13:21.308273 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be934bef-2220-45d2-b53d-21e360f10b0c admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:21.308567 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be934bef-2220-45d2-b53d-21e360f10b0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:21.308850 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be934bef-2220-45d2-b53d-21e360f10b0c admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:21.309592 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be934bef-2220-45d2-b53d-21e360f10b0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:13:21.311220 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be934bef-2220-45d2-b53d-21e360f10b0c admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:13:21.324939 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1239/2508] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 16:13:21 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:13:21.326788 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52839f9-a0b5-457e-b45d-6a79218303a0 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:13:21.337718 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52839f9-a0b5-457e-b45d-6a79218303a0 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:13:21.342493 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca2ec70-6b1c-4e34-a85f-994475662d9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:13:21.343168 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca2ec70-6b1c-4e34-a85f-994475662d9a admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:21.343458 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca2ec70-6b1c-4e34-a85f-994475662d9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:21.343773 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca2ec70-6b1c-4e34-a85f-994475662d9a admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:21.344508 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca2ec70-6b1c-4e34-a85f-994475662d9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:13:21.346125 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca2ec70-6b1c-4e34-a85f-994475662d9a admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:13:21.357890 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1270/2509] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 16:13:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:13:21.362010 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be934bef-2220-45d2-b53d-21e360f10b0c admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:13:21.373848 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be934bef-2220-45d2-b53d-21e360f10b0c admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:13:21.379164 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7492d16-952f-4de5-800b-de3809b573af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:13:21.380102 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7492d16-952f-4de5-800b-de3809b573af admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:21.380488 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7492d16-952f-4de5-800b-de3809b573af admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:21.380777 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7492d16-952f-4de5-800b-de3809b573af admin admin] PATH_INFO: `/v3/projects/00f0e8d5e82143e9b467a2a828174003/users/03ee2c5ef4554b31a497c44c204f810b/roles/b8c98fddc0fd4436b577fd1d3361bc38` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:21.394419 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:13:21.398832 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7492d16-952f-4de5-800b-de3809b573af admin admin] RBAC: Authorizing `identity:create_grant(project_id=00f0e8d5e82143e9b467a2a828174003, user_id=03ee2c5ef4554b31a497c44c204f810b, role_id=b8c98fddc0fd4436b577fd1d3361bc38)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:13:21.402191 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7492d16-952f-4de5-800b-de3809b573af admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:13:21.416876 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1240/2510] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 16:13:21 2022] PUT /identity/v3/projects/00f0e8d5e82143e9b467a2a828174003/users/03ee2c5ef4554b31a497c44c204f810b/roles/b8c98fddc0fd4436b577fd1d3361bc38 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:13:21.428718 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dded9bac-4897-4be2-b30a-d148510e7fb4 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:21.429285 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dded9bac-4897-4be2-b30a-d148510e7fb4 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:21.429568 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dded9bac-4897-4be2-b30a-d148510e7fb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:21.457507 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-dded9bac-4897-4be2-b30a-d148510e7fb4 None None] MFA Rules not processed for user `03ee2c5ef4554b31a497c44c204f810b`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:13:21.458730 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-dded9bac-4897-4be2-b30a-d148510e7fb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:13:21.548764 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1271/2511] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 16:13:21 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:13:21.554033 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7492d16-952f-4de5-800b-de3809b573af admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:13:21.622148 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7492d16-952f-4de5-800b-de3809b573af admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:13:21.626791 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86c5454-fa0a-4412-a1d2-3db2fc86ab5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:13:21.627456 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86c5454-fa0a-4412-a1d2-3db2fc86ab5b admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:21.627780 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86c5454-fa0a-4412-a1d2-3db2fc86ab5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:21.628062 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86c5454-fa0a-4412-a1d2-3db2fc86ab5b admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:21.629419 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86c5454-fa0a-4412-a1d2-3db2fc86ab5b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:13:21.631431 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86c5454-fa0a-4412-a1d2-3db2fc86ab5b admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:13:21.659640 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1241/2512] 10.222.0.99 () {60 vars in 1152 bytes} [Thu Jul 14 16:13:21 2022] POST /identity/v3/projects => generated 405 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 16:13:21.664181 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dded9bac-4897-4be2-b30a-d148510e7fb4 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:13:21.676066 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dded9bac-4897-4be2-b30a-d148510e7fb4 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:13:21.705740 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:13:21.735854 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8e7fec-0191-47a1-9024-09ae7d1dd49a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:13:21.736610 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8e7fec-0191-47a1-9024-09ae7d1dd49a admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:21.736896 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8e7fec-0191-47a1-9024-09ae7d1dd49a admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:21.737174 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8e7fec-0191-47a1-9024-09ae7d1dd49a admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:21.737867 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8e7fec-0191-47a1-9024-09ae7d1dd49a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:13:21.740169 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8e7fec-0191-47a1-9024-09ae7d1dd49a admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:13:21.761855 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1272/2513] 10.222.0.99 () {60 vars in 1143 bytes} [Thu Jul 14 16:13:21 2022] POST /identity/v3/users => generated 309 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 16:13:21.766014 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86c5454-fa0a-4412-a1d2-3db2fc86ab5b admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:13:21.777923 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86c5454-fa0a-4412-a1d2-3db2fc86ab5b admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:13:21.782588 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93910c14-28f0-407a-ad45-5802fed3d3f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:13:21.783287 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93910c14-28f0-407a-ad45-5802fed3d3f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:21.783578 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93910c14-28f0-407a-ad45-5802fed3d3f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:21.783914 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93910c14-28f0-407a-ad45-5802fed3d3f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:21.784638 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93910c14-28f0-407a-ad45-5802fed3d3f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:13:21.786859 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93910c14-28f0-407a-ad45-5802fed3d3f4 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:13:21.798223 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1242/2514] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 16:13:21 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:13:21.802608 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8e7fec-0191-47a1-9024-09ae7d1dd49a admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:13:21.812694 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8e7fec-0191-47a1-9024-09ae7d1dd49a admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:13:21.818850 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a26f5ec-9f22-4988-a435-e14fd22f83c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:13:21.819804 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a26f5ec-9f22-4988-a435-e14fd22f83c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:21.820091 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a26f5ec-9f22-4988-a435-e14fd22f83c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:21.820376 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a26f5ec-9f22-4988-a435-e14fd22f83c8 admin admin] PATH_INFO: `/v3/projects/7975dabf395e426fb484889ca78bc026/users/fe3c6c701b9546f38c4e674bf9e7f4ad/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:21.839659 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a26f5ec-9f22-4988-a435-e14fd22f83c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7975dabf395e426fb484889ca78bc026, user_id=fe3c6c701b9546f38c4e674bf9e7f4ad, role_id=d0f6ba1314f14d9f9c816bdb5d0c3aac)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:13:21.842272 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a26f5ec-9f22-4988-a435-e14fd22f83c8 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:13:21.859250 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1273/2515] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 16:13:21 2022] PUT /identity/v3/projects/7975dabf395e426fb484889ca78bc026/users/fe3c6c701b9546f38c4e674bf9e7f4ad/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:13:21.860458 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93910c14-28f0-407a-ad45-5802fed3d3f4 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:13:21.870689 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93910c14-28f0-407a-ad45-5802fed3d3f4 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:13:21.946212 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f09944-3f45-4215-903a-0400bc9d4664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:13:21.947528 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f09944-3f45-4215-903a-0400bc9d4664 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:21.947528 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f09944-3f45-4215-903a-0400bc9d4664 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:21.947846 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f09944-3f45-4215-903a-0400bc9d4664 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:21.948573 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f09944-3f45-4215-903a-0400bc9d4664 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:13:21.951291 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f09944-3f45-4215-903a-0400bc9d4664 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:13:21.966053 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1243/2516] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 16:13:21 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:13:21.970928 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a26f5ec-9f22-4988-a435-e14fd22f83c8 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:13:21.992936 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a26f5ec-9f22-4988-a435-e14fd22f83c8 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:13:21.997999 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3feb95-8e95-4329-a14f-bed4099c0853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:13:21.998906 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3feb95-8e95-4329-a14f-bed4099c0853 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:21.999189 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3feb95-8e95-4329-a14f-bed4099c0853 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:21.999469 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3feb95-8e95-4329-a14f-bed4099c0853 admin admin] PATH_INFO: `/v3/projects/7975dabf395e426fb484889ca78bc026/users/fe3c6c701b9546f38c4e674bf9e7f4ad/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:22.001554 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3feb95-8e95-4329-a14f-bed4099c0853 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7975dabf395e426fb484889ca78bc026, user_id=fe3c6c701b9546f38c4e674bf9e7f4ad, role_id=d0f6ba1314f14d9f9c816bdb5d0c3aac)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:13:22.005188 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3feb95-8e95-4329-a14f-bed4099c0853 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:13:22.016932 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1274/2517] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 16:13:21 2022] PUT /identity/v3/projects/7975dabf395e426fb484889ca78bc026/users/fe3c6c701b9546f38c4e674bf9e7f4ad/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:13:22.028808 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81600c31-1c0b-478a-9206-9cc6a07b98aa None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:22.029126 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81600c31-1c0b-478a-9206-9cc6a07b98aa None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:22.029391 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81600c31-1c0b-478a-9206-9cc6a07b98aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:22.073107 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-81600c31-1c0b-478a-9206-9cc6a07b98aa None None] MFA Rules not processed for user `fe3c6c701b9546f38c4e674bf9e7f4ad`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:13:22.074350 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-81600c31-1c0b-478a-9206-9cc6a07b98aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:13:22.142536 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:13:22.173366 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1244/2518] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 16:13:22 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:13:22.184387 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3feb95-8e95-4329-a14f-bed4099c0853 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:13:22.186454 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-ee3feb95-8e95-4329-a14f-bed4099c0853 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:13:22.258333 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3feb95-8e95-4329-a14f-bed4099c0853 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:13:22.265465 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ecffd6-8307-46ed-b85b-50e59bcbb2d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3e1a2b83f3d4d88ba5084dfaffd22f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:13:22.266398 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ecffd6-8307-46ed-b85b-50e59bcbb2d7 service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:22.266851 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ecffd6-8307-46ed-b85b-50e59bcbb2d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:22.267440 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ecffd6-8307-46ed-b85b-50e59bcbb2d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:22.281337 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ecffd6-8307-46ed-b85b-50e59bcbb2d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:13:22.282802 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ecffd6-8307-46ed-b85b-50e59bcbb2d7 service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:13:22.294808 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1275/2519] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:13:22 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:13:22.435778 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81600c31-1c0b-478a-9206-9cc6a07b98aa None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:13:22.507634 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81600c31-1c0b-478a-9206-9cc6a07b98aa None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:13:22.512179 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78fe0a3-439b-4ef5-b587-1fb4b8adb148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:13:22.513069 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78fe0a3-439b-4ef5-b587-1fb4b8adb148 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:22.513355 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78fe0a3-439b-4ef5-b587-1fb4b8adb148 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:22.513634 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78fe0a3-439b-4ef5-b587-1fb4b8adb148 admin admin] PATH_INFO: `/v3/users/03ee2c5ef4554b31a497c44c204f810b` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:22.514843 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78fe0a3-439b-4ef5-b587-1fb4b8adb148 admin admin] RBAC: Authorizing `identity:delete_user(user_id=03ee2c5ef4554b31a497c44c204f810b)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:13:22.516830 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78fe0a3-439b-4ef5-b587-1fb4b8adb148 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:13:22.578045 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-b78fe0a3-439b-4ef5-b587-1fb4b8adb148 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03ee2c5ef4554b31a497c44c204f810b'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:13:22.588985 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-b78fe0a3-439b-4ef5-b587-1fb4b8adb148 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03ee2c5ef4554b31a497c44c204f810b'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:13:22.603312 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-b78fe0a3-439b-4ef5-b587-1fb4b8adb148 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03ee2c5ef4554b31a497c44c204f810b'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:13:22.603988 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-b78fe0a3-439b-4ef5-b587-1fb4b8adb148 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03ee2c5ef4554b31a497c44c204f810b'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:13:22.604590 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-b78fe0a3-439b-4ef5-b587-1fb4b8adb148 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03ee2c5ef4554b31a497c44c204f810b'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:13:22.618511 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1245/2520] 10.222.0.99 () {58 vars in 1223 bytes} [Thu Jul 14 16:13:22 2022] DELETE /identity/v3/users/03ee2c5ef4554b31a497c44c204f810b => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:13:22.622952 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63441d8d-fe51-442d-b81f-84b199092431 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:22.623344 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63441d8d-fe51-442d-b81f-84b199092431 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:22.623739 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63441d8d-fe51-442d-b81f-84b199092431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:22.643114 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-63441d8d-fe51-442d-b81f-84b199092431 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:13:22.644540 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-63441d8d-fe51-442d-b81f-84b199092431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:13:22.707177 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1276/2521] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 16:13:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 16:13:22.716922 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78fe0a3-439b-4ef5-b587-1fb4b8adb148 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:13:22.718452 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b78fe0a3-439b-4ef5-b587-1fb4b8adb148 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:13:22.778508 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78fe0a3-439b-4ef5-b587-1fb4b8adb148 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:13:22.782468 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2390b87f-87d0-4922-b500-6a9103c183c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:13:22.783127 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2390b87f-87d0-4922-b500-6a9103c183c8 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:22.783421 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2390b87f-87d0-4922-b500-6a9103c183c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:22.783763 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2390b87f-87d0-4922-b500-6a9103c183c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:22.793258 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2390b87f-87d0-4922-b500-6a9103c183c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:13:22.794704 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2390b87f-87d0-4922-b500-6a9103c183c8 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:13:22.805769 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1246/2522] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:13:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 16:13:23.115541 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63441d8d-fe51-442d-b81f-84b199092431 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:13:23.117525 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-63441d8d-fe51-442d-b81f-84b199092431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:13:23.131569 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63441d8d-fe51-442d-b81f-84b199092431 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:13:23.137893 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27a18e0-da79-4862-9dd5-69847789514a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:13:23.138546 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27a18e0-da79-4862-9dd5-69847789514a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:23.138906 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27a18e0-da79-4862-9dd5-69847789514a admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:23.139254 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27a18e0-da79-4862-9dd5-69847789514a admin admin] PATH_INFO: `/v3/projects/00f0e8d5e82143e9b467a2a828174003` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:23.141861 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27a18e0-da79-4862-9dd5-69847789514a admin admin] RBAC: Authorizing `identity:delete_project(project_id=00f0e8d5e82143e9b467a2a828174003)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:13:23.142956 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27a18e0-da79-4862-9dd5-69847789514a admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:13:23.171519 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:13:23.200938 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-b27a18e0-da79-4862-9dd5-69847789514a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00f0e8d5e82143e9b467a2a828174003', 'request_id': 'req-b27a18e0-da79-4862-9dd5-69847789514a'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:13:23.208057 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-b27a18e0-da79-4862-9dd5-69847789514a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00f0e8d5e82143e9b467a2a828174003', 'request_id': 'req-b27a18e0-da79-4862-9dd5-69847789514a'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:13:23.214832 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-b27a18e0-da79-4862-9dd5-69847789514a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00f0e8d5e82143e9b467a2a828174003', 'request_id': 'req-b27a18e0-da79-4862-9dd5-69847789514a'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:13:23.230838 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-b27a18e0-da79-4862-9dd5-69847789514a admin admin] The token cache is being invalidate because project 00f0e8d5e82143e9b467a2a828174003 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 16:13:23.231428 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-b27a18e0-da79-4862-9dd5-69847789514a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:13:23.232111 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-b27a18e0-da79-4862-9dd5-69847789514a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:13:23.233931 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1277/2523] 10.222.0.99 () {58 vars in 1232 bytes} [Thu Jul 14 16:13:23 2022] DELETE /identity/v3/projects/00f0e8d5e82143e9b467a2a828174003 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:13:23.238653 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2390b87f-87d0-4922-b500-6a9103c183c8 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:13:23.249823 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2390b87f-87d0-4922-b500-6a9103c183c8 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:13:23.259195 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:13:23.322133 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717f6d3b-2fe7-4796-bbe7-7913c4b70515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:13:23.323024 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717f6d3b-2fe7-4796-bbe7-7913c4b70515 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:23.323400 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717f6d3b-2fe7-4796-bbe7-7913c4b70515 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:23.323796 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717f6d3b-2fe7-4796-bbe7-7913c4b70515 admin admin] PATH_INFO: `/v3/users/fe3c6c701b9546f38c4e674bf9e7f4ad` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:23.325156 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717f6d3b-2fe7-4796-bbe7-7913c4b70515 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe3c6c701b9546f38c4e674bf9e7f4ad)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:13:23.327157 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717f6d3b-2fe7-4796-bbe7-7913c4b70515 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:13:23.379873 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-717f6d3b-2fe7-4796-bbe7-7913c4b70515 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe3c6c701b9546f38c4e674bf9e7f4ad'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:13:23.388440 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-717f6d3b-2fe7-4796-bbe7-7913c4b70515 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe3c6c701b9546f38c4e674bf9e7f4ad'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:13:23.400972 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-717f6d3b-2fe7-4796-bbe7-7913c4b70515 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe3c6c701b9546f38c4e674bf9e7f4ad'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:13:23.401641 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-717f6d3b-2fe7-4796-bbe7-7913c4b70515 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe3c6c701b9546f38c4e674bf9e7f4ad'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:13:23.402299 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-717f6d3b-2fe7-4796-bbe7-7913c4b70515 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe3c6c701b9546f38c4e674bf9e7f4ad'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:13:23.415489 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1247/2524] 10.222.0.99 () {58 vars in 1223 bytes} [Thu Jul 14 16:13:23 2022] DELETE /identity/v3/users/fe3c6c701b9546f38c4e674bf9e7f4ad => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:13:23.709452 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27a18e0-da79-4862-9dd5-69847789514a admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:13:23.711220 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-b27a18e0-da79-4862-9dd5-69847789514a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:13:23.725532 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27a18e0-da79-4862-9dd5-69847789514a admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:13:23.796452 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6714a17-df79-4ded-8daa-005e01908cf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:13:23.797324 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6714a17-df79-4ded-8daa-005e01908cf4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:23.797612 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6714a17-df79-4ded-8daa-005e01908cf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:23.797889 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6714a17-df79-4ded-8daa-005e01908cf4 admin admin] PATH_INFO: `/v3/projects/7975dabf395e426fb484889ca78bc026` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:23.799083 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6714a17-df79-4ded-8daa-005e01908cf4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7975dabf395e426fb484889ca78bc026)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:13:23.801373 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6714a17-df79-4ded-8daa-005e01908cf4 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:13:23.848393 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-e6714a17-df79-4ded-8daa-005e01908cf4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7975dabf395e426fb484889ca78bc026', 'request_id': 'req-e6714a17-df79-4ded-8daa-005e01908cf4'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:13:23.855806 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-e6714a17-df79-4ded-8daa-005e01908cf4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7975dabf395e426fb484889ca78bc026', 'request_id': 'req-e6714a17-df79-4ded-8daa-005e01908cf4'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:13:23.861482 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-e6714a17-df79-4ded-8daa-005e01908cf4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7975dabf395e426fb484889ca78bc026', 'request_id': 'req-e6714a17-df79-4ded-8daa-005e01908cf4'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:13:23.881737 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-e6714a17-df79-4ded-8daa-005e01908cf4 admin admin] The token cache is being invalidate because project 7975dabf395e426fb484889ca78bc026 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 16:13:23.881962 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-e6714a17-df79-4ded-8daa-005e01908cf4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:13:23.882581 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-e6714a17-df79-4ded-8daa-005e01908cf4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:13:23.884163 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1278/2525] 10.222.0.99 () {58 vars in 1232 bytes} [Thu Jul 14 16:13:23 2022] DELETE /identity/v3/projects/7975dabf395e426fb484889ca78bc026 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:13:23.889185 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc08eec-1569-4e89-9086-7f74dd9c6a24 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:23.889684 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc08eec-1569-4e89-9086-7f74dd9c6a24 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:23.889801 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc08eec-1569-4e89-9086-7f74dd9c6a24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:23.913473 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-fbc08eec-1569-4e89-9086-7f74dd9c6a24 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:13:23.914712 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-fbc08eec-1569-4e89-9086-7f74dd9c6a24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:13:23.976470 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1248/2526] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 16:13:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:13:23.984227 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a190f53-4ed9-4f97-a8d4-e90c347b75d0 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:23.984525 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a190f53-4ed9-4f97-a8d4-e90c347b75d0 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:23.984812 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a190f53-4ed9-4f97-a8d4-e90c347b75d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:24.004107 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-0a190f53-4ed9-4f97-a8d4-e90c347b75d0 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:13:24.005367 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-0a190f53-4ed9-4f97-a8d4-e90c347b75d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:13:24.013365 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1279/2527] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 16:13:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:13:24.017569 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc08eec-1569-4e89-9086-7f74dd9c6a24 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:13:24.028344 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc08eec-1569-4e89-9086-7f74dd9c6a24 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:13:24.032890 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b68db5-4e84-4a1b-bcc5-fff02821921c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:13:24.033549 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b68db5-4e84-4a1b-bcc5-fff02821921c admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:24.033838 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b68db5-4e84-4a1b-bcc5-fff02821921c admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:24.034121 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b68db5-4e84-4a1b-bcc5-fff02821921c admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:24.035607 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b68db5-4e84-4a1b-bcc5-fff02821921c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 16:13:24.036267 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b68db5-4e84-4a1b-bcc5-fff02821921c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:13:24.037891 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b68db5-4e84-4a1b-bcc5-fff02821921c admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:13:24.049747 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1249/2528] 10.222.0.99 () {58 vars in 1152 bytes} [Thu Jul 14 16:13:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 16:13:24.054316 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a190f53-4ed9-4f97-a8d4-e90c347b75d0 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:13:24.063761 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a190f53-4ed9-4f97-a8d4-e90c347b75d0 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:13:24.069167 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9384f3cd-0d2a-4741-afa4-b672a3811a24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:13:24.069979 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9384f3cd-0d2a-4741-afa4-b672a3811a24 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:24.070162 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9384f3cd-0d2a-4741-afa4-b672a3811a24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:24.070443 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9384f3cd-0d2a-4741-afa4-b672a3811a24 admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:24.071965 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9384f3cd-0d2a-4741-afa4-b672a3811a24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:13:24.073313 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9384f3cd-0d2a-4741-afa4-b672a3811a24 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:13:24.100083 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1280/2529] 10.222.0.99 () {60 vars in 1152 bytes} [Thu Jul 14 16:13:24 2022] POST /identity/v3/projects => generated 355 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 16:13:24.104329 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b68db5-4e84-4a1b-bcc5-fff02821921c admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:13:24.115506 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b68db5-4e84-4a1b-bcc5-fff02821921c admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:13:24.127299 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:13:24.173947 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ddfee4-dfe4-4ed9-9b21-91a2922fa371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:13:24.174612 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ddfee4-dfe4-4ed9-9b21-91a2922fa371 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:24.174897 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ddfee4-dfe4-4ed9-9b21-91a2922fa371 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:24.175175 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ddfee4-dfe4-4ed9-9b21-91a2922fa371 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:24.176742 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ddfee4-dfe4-4ed9-9b21-91a2922fa371 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:13:24.179392 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ddfee4-dfe4-4ed9-9b21-91a2922fa371 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:13:24.207649 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1250/2530] 10.222.0.99 () {60 vars in 1143 bytes} [Thu Jul 14 16:13:24 2022] POST /identity/v3/users => generated 278 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 16:13:24.210005 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9384f3cd-0d2a-4741-afa4-b672a3811a24 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:13:24.226501 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9384f3cd-0d2a-4741-afa4-b672a3811a24 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:13:24.231883 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d729e2-f953-46f9-a7a6-87a1b6701c47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:13:24.232566 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d729e2-f953-46f9-a7a6-87a1b6701c47 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:24.232855 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d729e2-f953-46f9-a7a6-87a1b6701c47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:24.233141 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d729e2-f953-46f9-a7a6-87a1b6701c47 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:24.233879 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d729e2-f953-46f9-a7a6-87a1b6701c47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:13:24.235499 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d729e2-f953-46f9-a7a6-87a1b6701c47 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:13:24.242061 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:13:24.250636 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1281/2531] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 16:13:24 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:13:24.252086 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ddfee4-dfe4-4ed9-9b21-91a2922fa371 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:13:24.266995 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ddfee4-dfe4-4ed9-9b21-91a2922fa371 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:13:24.272285 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab99b9b-f63e-4996-ae76-e9805f57dfb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:13:24.273001 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab99b9b-f63e-4996-ae76-e9805f57dfb1 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:24.273292 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab99b9b-f63e-4996-ae76-e9805f57dfb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:24.273576 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab99b9b-f63e-4996-ae76-e9805f57dfb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:24.274321 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab99b9b-f63e-4996-ae76-e9805f57dfb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:13:24.276045 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab99b9b-f63e-4996-ae76-e9805f57dfb1 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:13:24.288692 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1251/2532] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 16:13:24 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 16:13:24.293743 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d729e2-f953-46f9-a7a6-87a1b6701c47 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:13:24.305512 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d729e2-f953-46f9-a7a6-87a1b6701c47 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:13:24.310553 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9fd444-93f4-4d26-9947-b22ee6c5539e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:13:24.311445 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9fd444-93f4-4d26-9947-b22ee6c5539e admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:24.311770 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9fd444-93f4-4d26-9947-b22ee6c5539e admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:24.312056 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9fd444-93f4-4d26-9947-b22ee6c5539e admin admin] PATH_INFO: `/v3/projects/d5b285bdeebb4c949286a414f34e45f0/users/6f6430a3a7164db2b39c2f8b90560503/roles/b8c98fddc0fd4436b577fd1d3361bc38` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:24.327568 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9fd444-93f4-4d26-9947-b22ee6c5539e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5b285bdeebb4c949286a414f34e45f0, user_id=6f6430a3a7164db2b39c2f8b90560503, role_id=b8c98fddc0fd4436b577fd1d3361bc38)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:13:24.330208 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9fd444-93f4-4d26-9947-b22ee6c5539e admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:13:24.343330 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1282/2533] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 16:13:24 2022] PUT /identity/v3/projects/d5b285bdeebb4c949286a414f34e45f0/users/6f6430a3a7164db2b39c2f8b90560503/roles/b8c98fddc0fd4436b577fd1d3361bc38 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:13:24.354406 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2780522-5334-4836-8a30-54474424c4e7 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:24.354697 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2780522-5334-4836-8a30-54474424c4e7 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:24.355018 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2780522-5334-4836-8a30-54474424c4e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:24.382058 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-d2780522-5334-4836-8a30-54474424c4e7 None None] MFA Rules not processed for user `6f6430a3a7164db2b39c2f8b90560503`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:13:24.383280 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d2780522-5334-4836-8a30-54474424c4e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:13:24.455352 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1252/2534] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 16:13:24 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:13:24.461271 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9fd444-93f4-4d26-9947-b22ee6c5539e admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:13:24.531732 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9fd444-93f4-4d26-9947-b22ee6c5539e admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:13:24.537236 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d60ea9d-5d25-4dd3-a2b1-761b6472a250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:13:24.538007 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d60ea9d-5d25-4dd3-a2b1-761b6472a250 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:24.538295 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d60ea9d-5d25-4dd3-a2b1-761b6472a250 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:24.538578 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d60ea9d-5d25-4dd3-a2b1-761b6472a250 admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:24.539275 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d60ea9d-5d25-4dd3-a2b1-761b6472a250 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:13:24.541200 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d60ea9d-5d25-4dd3-a2b1-761b6472a250 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:13:24.567149 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1283/2535] 10.222.0.99 () {60 vars in 1152 bytes} [Thu Jul 14 16:13:24 2022] POST /identity/v3/projects => generated 357 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 16:13:24.571194 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2780522-5334-4836-8a30-54474424c4e7 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:13:24.582877 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2780522-5334-4836-8a30-54474424c4e7 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:13:24.646963 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18787a0d-659a-4f34-ac85-a808f53b12d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:13:24.647665 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18787a0d-659a-4f34-ac85-a808f53b12d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:24.647954 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18787a0d-659a-4f34-ac85-a808f53b12d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:24.648238 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18787a0d-659a-4f34-ac85-a808f53b12d8 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:24.648957 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18787a0d-659a-4f34-ac85-a808f53b12d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:13:24.650866 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18787a0d-659a-4f34-ac85-a808f53b12d8 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:13:24.671687 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1253/2536] 10.222.0.99 () {60 vars in 1143 bytes} [Thu Jul 14 16:13:24 2022] POST /identity/v3/users => generated 285 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 16:13:24.675809 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d60ea9d-5d25-4dd3-a2b1-761b6472a250 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:13:24.686789 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d60ea9d-5d25-4dd3-a2b1-761b6472a250 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:13:24.691510 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a66806f-d593-48fe-b91e-66a312f5c617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:13:24.692291 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a66806f-d593-48fe-b91e-66a312f5c617 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:24.692677 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a66806f-d593-48fe-b91e-66a312f5c617 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:24.693027 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a66806f-d593-48fe-b91e-66a312f5c617 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:24.693781 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a66806f-d593-48fe-b91e-66a312f5c617 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:13:24.695396 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a66806f-d593-48fe-b91e-66a312f5c617 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:13:24.707622 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1284/2537] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 16:13:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:13:24.712438 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18787a0d-659a-4f34-ac85-a808f53b12d8 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:13:24.723071 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18787a0d-659a-4f34-ac85-a808f53b12d8 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:13:24.727830 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4edcc52-5688-4bbe-ba3b-2d08dfc4fd84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:13:24.728809 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4edcc52-5688-4bbe-ba3b-2d08dfc4fd84 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:24.729089 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4edcc52-5688-4bbe-ba3b-2d08dfc4fd84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:24.729826 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4edcc52-5688-4bbe-ba3b-2d08dfc4fd84 admin admin] PATH_INFO: `/v3/projects/6611144c16564379a5c1fb49869285ef/users/8bcd44a40ec942adaf2b705000ec7b49/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:24.741792 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:13:24.746737 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4edcc52-5688-4bbe-ba3b-2d08dfc4fd84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6611144c16564379a5c1fb49869285ef, user_id=8bcd44a40ec942adaf2b705000ec7b49, role_id=d0f6ba1314f14d9f9c816bdb5d0c3aac)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:13:24.749895 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4edcc52-5688-4bbe-ba3b-2d08dfc4fd84 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:13:24.762397 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1254/2538] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 16:13:24 2022] PUT /identity/v3/projects/6611144c16564379a5c1fb49869285ef/users/8bcd44a40ec942adaf2b705000ec7b49/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:13:24.766345 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a66806f-d593-48fe-b91e-66a312f5c617 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:13:24.777329 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a66806f-d593-48fe-b91e-66a312f5c617 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:13:24.838848 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbbdab0-e097-403b-bafd-5f2a95ea3217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:13:24.839554 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbbdab0-e097-403b-bafd-5f2a95ea3217 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:24.839880 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbbdab0-e097-403b-bafd-5f2a95ea3217 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:24.840157 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbbdab0-e097-403b-bafd-5f2a95ea3217 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:24.840887 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbbdab0-e097-403b-bafd-5f2a95ea3217 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:13:24.842478 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbbdab0-e097-403b-bafd-5f2a95ea3217 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:13:24.852326 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1285/2539] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 16:13:24 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:13:24.856507 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4edcc52-5688-4bbe-ba3b-2d08dfc4fd84 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:13:24.867344 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4edcc52-5688-4bbe-ba3b-2d08dfc4fd84 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:13:24.871993 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa819a77-0517-4de2-866a-33ad854b868e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:13:24.872875 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa819a77-0517-4de2-866a-33ad854b868e admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:24.873164 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa819a77-0517-4de2-866a-33ad854b868e admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:24.873447 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa819a77-0517-4de2-866a-33ad854b868e admin admin] PATH_INFO: `/v3/projects/6611144c16564379a5c1fb49869285ef/users/8bcd44a40ec942adaf2b705000ec7b49/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:24.875413 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa819a77-0517-4de2-866a-33ad854b868e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6611144c16564379a5c1fb49869285ef, user_id=8bcd44a40ec942adaf2b705000ec7b49, role_id=d0f6ba1314f14d9f9c816bdb5d0c3aac)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:13:24.878083 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa819a77-0517-4de2-866a-33ad854b868e admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:13:24.889184 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1255/2540] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 16:13:24 2022] PUT /identity/v3/projects/6611144c16564379a5c1fb49869285ef/users/8bcd44a40ec942adaf2b705000ec7b49/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:13:24.900559 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d582ead7-d167-4fdb-ac73-533bd48900d4 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:24.900859 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d582ead7-d167-4fdb-ac73-533bd48900d4 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:24.901146 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d582ead7-d167-4fdb-ac73-533bd48900d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:24.927541 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-d582ead7-d167-4fdb-ac73-533bd48900d4 None None] MFA Rules not processed for user `8bcd44a40ec942adaf2b705000ec7b49`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:13:24.928798 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-d582ead7-d167-4fdb-ac73-533bd48900d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:13:24.964523 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:13:25.000898 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1286/2541] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 16:13:24 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:13:25.013314 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa819a77-0517-4de2-866a-33ad854b868e admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:13:25.015072 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-aa819a77-0517-4de2-866a-33ad854b868e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:13:25.072493 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa819a77-0517-4de2-866a-33ad854b868e admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:13:25.076124 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7e3aac-be63-4bcb-9a1a-ba4ce850fb9a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3e1a2b83f3d4d88ba5084dfaffd22f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:13:25.076799 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7e3aac-be63-4bcb-9a1a-ba4ce850fb9a service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:25.077088 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7e3aac-be63-4bcb-9a1a-ba4ce850fb9a service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:25.077369 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7e3aac-be63-4bcb-9a1a-ba4ce850fb9a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:25.086914 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7e3aac-be63-4bcb-9a1a-ba4ce850fb9a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:13:25.088378 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7e3aac-be63-4bcb-9a1a-ba4ce850fb9a service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:13:25.097707 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1256/2542] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:13:25 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:13:25.168297 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d582ead7-d167-4fdb-ac73-533bd48900d4 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:13:25.179192 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d582ead7-d167-4fdb-ac73-533bd48900d4 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:13:25.182823 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c45eb5-f283-4ba4-9aa4-8e322f74e1d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3e1a2b83f3d4d88ba5084dfaffd22f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:13:25.183491 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c45eb5-f283-4ba4-9aa4-8e322f74e1d1 service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:25.183800 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c45eb5-f283-4ba4-9aa4-8e322f74e1d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:25.184081 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c45eb5-f283-4ba4-9aa4-8e322f74e1d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:25.244129 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c45eb5-f283-4ba4-9aa4-8e322f74e1d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:13:25.245498 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c45eb5-f283-4ba4-9aa4-8e322f74e1d1 service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:13:25.255526 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1287/2543] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:13:25 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:13:25.564847 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7e3aac-be63-4bcb-9a1a-ba4ce850fb9a service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:13:25.566876 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ef7e3aac-be63-4bcb-9a1a-ba4ce850fb9a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:13:25.624337 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7e3aac-be63-4bcb-9a1a-ba4ce850fb9a service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:13:25.628037 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e62ce53-da75-4164-8cbc-572ac4028bd4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:13:25.628985 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e62ce53-da75-4164-8cbc-572ac4028bd4 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:25.629400 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e62ce53-da75-4164-8cbc-572ac4028bd4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:25.629679 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e62ce53-da75-4164-8cbc-572ac4028bd4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:25.631474 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-3e62ce53-da75-4164-8cbc-572ac4028bd4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:13:25.709937 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e62ce53-da75-4164-8cbc-572ac4028bd4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:13:25.711468 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e62ce53-da75-4164-8cbc-572ac4028bd4 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:13:25.721329 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1257/2544] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:13:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:13:26.180928 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c45eb5-f283-4ba4-9aa4-8e322f74e1d1 service cinder] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:13:26.182396 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-e3c45eb5-f283-4ba4-9aa4-8e322f74e1d1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:13:26.241484 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e62ce53-da75-4164-8cbc-572ac4028bd4 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:13:26.243646 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-3e62ce53-da75-4164-8cbc-572ac4028bd4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:13:26.255942 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e62ce53-da75-4164-8cbc-572ac4028bd4 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:13:26.260595 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9bdb31-a445-4fef-a238-be2fddf490b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:13:26.261475 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9bdb31-a445-4fef-a238-be2fddf490b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:26.261760 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9bdb31-a445-4fef-a238-be2fddf490b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:26.262037 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9bdb31-a445-4fef-a238-be2fddf490b2 admin admin] PATH_INFO: `/v3/users/71bc270c6253426baa4f79638d466b20` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:26.262250 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c45eb5-f283-4ba4-9aa4-8e322f74e1d1 service cinder] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:13:26.263223 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9bdb31-a445-4fef-a238-be2fddf490b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=71bc270c6253426baa4f79638d466b20)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:13:26.265213 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9bdb31-a445-4fef-a238-be2fddf490b2 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:13:26.267330 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704dcec6-ef46-4e92-b312-70d5ddf8cc4f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc157c485f794e968a60d8405c9310ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:13:26.268020 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704dcec6-ef46-4e92-b312-70d5ddf8cc4f service placement] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:26.268311 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704dcec6-ef46-4e92-b312-70d5ddf8cc4f service placement] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:26.268593 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704dcec6-ef46-4e92-b312-70d5ddf8cc4f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:26.270959 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-704dcec6-ef46-4e92-b312-70d5ddf8cc4f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:13:26.286398 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704dcec6-ef46-4e92-b312-70d5ddf8cc4f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:13:26.293076 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704dcec6-ef46-4e92-b312-70d5ddf8cc4f service placement] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:13:26.294179 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:13:26.305565 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1288/2545] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:13:26 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 16:13:26.332644 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4b9bdb31-a445-4fef-a238-be2fddf490b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71bc270c6253426baa4f79638d466b20'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:13:26.344988 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4b9bdb31-a445-4fef-a238-be2fddf490b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71bc270c6253426baa4f79638d466b20'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:13:26.360794 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4b9bdb31-a445-4fef-a238-be2fddf490b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71bc270c6253426baa4f79638d466b20'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:13:26.361570 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4b9bdb31-a445-4fef-a238-be2fddf490b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71bc270c6253426baa4f79638d466b20'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:13:26.362215 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4b9bdb31-a445-4fef-a238-be2fddf490b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71bc270c6253426baa4f79638d466b20'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:13:26.374969 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1258/2546] 10.222.0.99 () {58 vars in 1223 bytes} [Thu Jul 14 16:13:26 2022] DELETE /identity/v3/users/71bc270c6253426baa4f79638d466b20 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 16:13:26.570782 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704dcec6-ef46-4e92-b312-70d5ddf8cc4f service placement] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:13:26.582555 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704dcec6-ef46-4e92-b312-70d5ddf8cc4f service placement] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:13:26.642306 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33b5f22-6393-4c02-807c-5d41994e27a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:13:26.643179 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33b5f22-6393-4c02-807c-5d41994e27a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:26.643475 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33b5f22-6393-4c02-807c-5d41994e27a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:26.643792 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33b5f22-6393-4c02-807c-5d41994e27a3 admin admin] PATH_INFO: `/v3/projects/858bb49b6e1e48568e2055eacc1ee7e5` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:26.645760 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33b5f22-6393-4c02-807c-5d41994e27a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=858bb49b6e1e48568e2055eacc1ee7e5)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:13:26.647750 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33b5f22-6393-4c02-807c-5d41994e27a3 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:13:26.649238 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:13:26.696626 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-c33b5f22-6393-4c02-807c-5d41994e27a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '858bb49b6e1e48568e2055eacc1ee7e5', 'request_id': 'req-c33b5f22-6393-4c02-807c-5d41994e27a3'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:13:26.702679 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-c33b5f22-6393-4c02-807c-5d41994e27a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '858bb49b6e1e48568e2055eacc1ee7e5', 'request_id': 'req-c33b5f22-6393-4c02-807c-5d41994e27a3'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:13:26.708883 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-c33b5f22-6393-4c02-807c-5d41994e27a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '858bb49b6e1e48568e2055eacc1ee7e5', 'request_id': 'req-c33b5f22-6393-4c02-807c-5d41994e27a3'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:13:26.721154 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-c33b5f22-6393-4c02-807c-5d41994e27a3 admin admin] The token cache is being invalidate because project 858bb49b6e1e48568e2055eacc1ee7e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 16:13:26.721449 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-c33b5f22-6393-4c02-807c-5d41994e27a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:13:26.722076 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-c33b5f22-6393-4c02-807c-5d41994e27a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:13:26.723676 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1289/2547] 10.222.0.99 () {58 vars in 1232 bytes} [Thu Jul 14 16:13:26 2022] DELETE /identity/v3/projects/858bb49b6e1e48568e2055eacc1ee7e5 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:13:26.728660 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea2cb93-ccc4-4f96-a622-4ee272d0c7ef None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:26.728954 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea2cb93-ccc4-4f96-a622-4ee272d0c7ef None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:26.729238 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea2cb93-ccc4-4f96-a622-4ee272d0c7ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:26.747953 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-0ea2cb93-ccc4-4f96-a622-4ee272d0c7ef None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:13:26.749164 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0ea2cb93-ccc4-4f96-a622-4ee272d0c7ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:13:26.810550 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1259/2548] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 16:13:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:13:26.818853 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7777052b-805b-43dc-8046-fc3b037f4bcd None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:26.819150 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7777052b-805b-43dc-8046-fc3b037f4bcd None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:26.819438 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7777052b-805b-43dc-8046-fc3b037f4bcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:26.837621 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-7777052b-805b-43dc-8046-fc3b037f4bcd None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:13:26.838864 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-7777052b-805b-43dc-8046-fc3b037f4bcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:13:26.846826 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1290/2549] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 16:13:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:13:26.851344 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea2cb93-ccc4-4f96-a622-4ee272d0c7ef None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:13:26.860957 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea2cb93-ccc4-4f96-a622-4ee272d0c7ef None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:13:26.866279 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b132e9-34b7-484f-bcda-2ec33b9a2b88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:13:26.866934 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b132e9-34b7-484f-bcda-2ec33b9a2b88 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:26.867222 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b132e9-34b7-484f-bcda-2ec33b9a2b88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:26.867504 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b132e9-34b7-484f-bcda-2ec33b9a2b88 admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:26.868262 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b132e9-34b7-484f-bcda-2ec33b9a2b88 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 16:13:26.868569 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b132e9-34b7-484f-bcda-2ec33b9a2b88 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:13:26.870170 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b132e9-34b7-484f-bcda-2ec33b9a2b88 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:13:26.882749 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1260/2550] 10.222.0.99 () {58 vars in 1152 bytes} [Thu Jul 14 16:13:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 16:13:26.887242 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7777052b-805b-43dc-8046-fc3b037f4bcd None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:13:26.897221 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7777052b-805b-43dc-8046-fc3b037f4bcd None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:13:26.901936 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3492734-7404-4d2e-b085-5d418dbe4c6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:13:26.902594 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3492734-7404-4d2e-b085-5d418dbe4c6f admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:26.902884 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3492734-7404-4d2e-b085-5d418dbe4c6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:26.903164 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3492734-7404-4d2e-b085-5d418dbe4c6f admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:26.903899 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3492734-7404-4d2e-b085-5d418dbe4c6f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:13:26.905809 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3492734-7404-4d2e-b085-5d418dbe4c6f admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:13:26.931887 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1291/2551] 10.222.0.99 () {60 vars in 1152 bytes} [Thu Jul 14 16:13:26 2022] POST /identity/v3/projects => generated 399 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 16:13:26.936115 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b132e9-34b7-484f-bcda-2ec33b9a2b88 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:13:26.946901 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b132e9-34b7-484f-bcda-2ec33b9a2b88 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:13:27.004912 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e11d720-8edf-4755-bd73-19f27fc6a553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:13:27.005594 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e11d720-8edf-4755-bd73-19f27fc6a553 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:27.005890 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e11d720-8edf-4755-bd73-19f27fc6a553 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:27.006176 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e11d720-8edf-4755-bd73-19f27fc6a553 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:27.006889 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e11d720-8edf-4755-bd73-19f27fc6a553 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:13:27.008858 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e11d720-8edf-4755-bd73-19f27fc6a553 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:13:27.029014 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1261/2552] 10.222.0.99 () {60 vars in 1143 bytes} [Thu Jul 14 16:13:26 2022] POST /identity/v3/users => generated 300 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 16:13:27.033070 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3492734-7404-4d2e-b085-5d418dbe4c6f admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:13:27.044100 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3492734-7404-4d2e-b085-5d418dbe4c6f admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:13:27.049021 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9192d63e-1d86-42ed-93b0-1537470044be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:13:27.049687 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9192d63e-1d86-42ed-93b0-1537470044be admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:27.049976 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9192d63e-1d86-42ed-93b0-1537470044be admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:27.050259 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9192d63e-1d86-42ed-93b0-1537470044be admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:27.051576 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9192d63e-1d86-42ed-93b0-1537470044be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:13:27.053237 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9192d63e-1d86-42ed-93b0-1537470044be admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:13:27.064134 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1292/2553] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 16:13:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:13:27.068987 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e11d720-8edf-4755-bd73-19f27fc6a553 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:13:27.079565 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e11d720-8edf-4755-bd73-19f27fc6a553 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:13:27.084425 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879f9bdf-d6fa-45e6-b8a9-1ded98e44c83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:13:27.085086 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879f9bdf-d6fa-45e6-b8a9-1ded98e44c83 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:27.085379 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879f9bdf-d6fa-45e6-b8a9-1ded98e44c83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:27.085661 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879f9bdf-d6fa-45e6-b8a9-1ded98e44c83 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:27.086396 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879f9bdf-d6fa-45e6-b8a9-1ded98e44c83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:13:27.088037 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879f9bdf-d6fa-45e6-b8a9-1ded98e44c83 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:13:27.098945 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1262/2554] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 16:13:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:13:27.102643 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9192d63e-1d86-42ed-93b0-1537470044be admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:13:27.112550 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9192d63e-1d86-42ed-93b0-1537470044be admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:13:27.118256 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09ad864-5e83-4b9e-bdfe-fc1ecbf5bce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:13:27.119141 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09ad864-5e83-4b9e-bdfe-fc1ecbf5bce0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:27.119427 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09ad864-5e83-4b9e-bdfe-fc1ecbf5bce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:27.119741 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09ad864-5e83-4b9e-bdfe-fc1ecbf5bce0 admin admin] PATH_INFO: `/v3/projects/2bf56351e9704e01aa67d4108a915fe7/users/6a7fb71a2a9f40cabafdd4225299adbc/roles/b8c98fddc0fd4436b577fd1d3361bc38` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:27.135200 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09ad864-5e83-4b9e-bdfe-fc1ecbf5bce0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2bf56351e9704e01aa67d4108a915fe7, user_id=6a7fb71a2a9f40cabafdd4225299adbc, role_id=b8c98fddc0fd4436b577fd1d3361bc38)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:13:27.137891 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09ad864-5e83-4b9e-bdfe-fc1ecbf5bce0 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:13:27.151561 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1293/2555] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 16:13:27 2022] PUT /identity/v3/projects/2bf56351e9704e01aa67d4108a915fe7/users/6a7fb71a2a9f40cabafdd4225299adbc/roles/b8c98fddc0fd4436b577fd1d3361bc38 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:13:27.162555 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ebb818-11ff-4e3c-ba08-0aaa5b4846d9 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:27.162849 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ebb818-11ff-4e3c-ba08-0aaa5b4846d9 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:27.163133 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ebb818-11ff-4e3c-ba08-0aaa5b4846d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:27.176728 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:13:27.193380 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-71ebb818-11ff-4e3c-ba08-0aaa5b4846d9 None None] MFA Rules not processed for user `6a7fb71a2a9f40cabafdd4225299adbc`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:13:27.194619 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-71ebb818-11ff-4e3c-ba08-0aaa5b4846d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:13:27.262304 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1263/2556] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 16:13:27 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:13:27.273427 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09ad864-5e83-4b9e-bdfe-fc1ecbf5bce0 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:13:27.275201 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-d09ad864-5e83-4b9e-bdfe-fc1ecbf5bce0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:13:27.333396 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09ad864-5e83-4b9e-bdfe-fc1ecbf5bce0 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:13:27.337187 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730dce59-0544-470a-a92f-70b83a595d9d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3e1a2b83f3d4d88ba5084dfaffd22f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:13:27.337832 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730dce59-0544-470a-a92f-70b83a595d9d service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:27.338118 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730dce59-0544-470a-a92f-70b83a595d9d service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:27.338404 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730dce59-0544-470a-a92f-70b83a595d9d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:27.346630 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730dce59-0544-470a-a92f-70b83a595d9d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:13:27.348608 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730dce59-0544-470a-a92f-70b83a595d9d service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:13:27.357298 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1294/2557] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:13:27 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:13:35.433566 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ebb818-11ff-4e3c-ba08-0aaa5b4846d9 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:13:35.505665 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ebb818-11ff-4e3c-ba08-0aaa5b4846d9 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:13:35.510535 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496161eb-ee4d-49de-9634-b7403de445f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:13:35.511355 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496161eb-ee4d-49de-9634-b7403de445f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:35.511679 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496161eb-ee4d-49de-9634-b7403de445f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:35.511963 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496161eb-ee4d-49de-9634-b7403de445f1 admin admin] PATH_INFO: `/v3/users/6a7fb71a2a9f40cabafdd4225299adbc` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:35.513380 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496161eb-ee4d-49de-9634-b7403de445f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a7fb71a2a9f40cabafdd4225299adbc)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:13:35.515353 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496161eb-ee4d-49de-9634-b7403de445f1 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:13:35.585200 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-496161eb-ee4d-49de-9634-b7403de445f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a7fb71a2a9f40cabafdd4225299adbc'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:13:35.596536 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-496161eb-ee4d-49de-9634-b7403de445f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a7fb71a2a9f40cabafdd4225299adbc'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:13:35.616756 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-496161eb-ee4d-49de-9634-b7403de445f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a7fb71a2a9f40cabafdd4225299adbc'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:13:35.617491 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-496161eb-ee4d-49de-9634-b7403de445f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a7fb71a2a9f40cabafdd4225299adbc'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:13:35.618103 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-496161eb-ee4d-49de-9634-b7403de445f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a7fb71a2a9f40cabafdd4225299adbc'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:13:35.634680 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1264/2558] 10.222.0.99 () {58 vars in 1223 bytes} [Thu Jul 14 16:13:35 2022] DELETE /identity/v3/users/6a7fb71a2a9f40cabafdd4225299adbc => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:13:35.639072 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca4c366-e3d1-4798-8983-8d9b589c8574 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:35.639589 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca4c366-e3d1-4798-8983-8d9b589c8574 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:35.639826 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca4c366-e3d1-4798-8983-8d9b589c8574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:35.657253 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-2ca4c366-e3d1-4798-8983-8d9b589c8574 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:13:35.658458 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-2ca4c366-e3d1-4798-8983-8d9b589c8574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:13:35.685727 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:13:35.726356 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1295/2559] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 16:13:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:13:35.736192 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496161eb-ee4d-49de-9634-b7403de445f1 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:13:35.737587 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-496161eb-ee4d-49de-9634-b7403de445f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:13:35.757050 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:13:35.799994 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496161eb-ee4d-49de-9634-b7403de445f1 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:13:35.804219 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d951b91-0f21-401f-986c-6206fad05818 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:13:35.804589 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d951b91-0f21-401f-986c-6206fad05818 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:35.804879 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d951b91-0f21-401f-986c-6206fad05818 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:35.805161 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d951b91-0f21-401f-986c-6206fad05818 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:35.814754 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d951b91-0f21-401f-986c-6206fad05818 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:13:35.816243 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d951b91-0f21-401f-986c-6206fad05818 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:13:35.826418 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1265/2560] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:13:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:13:36.106305 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca4c366-e3d1-4798-8983-8d9b589c8574 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:13:36.107778 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-2ca4c366-e3d1-4798-8983-8d9b589c8574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:13:36.129487 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca4c366-e3d1-4798-8983-8d9b589c8574 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:13:36.134775 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e30ee46-58b0-4fb4-af37-bdcb932a80d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:13:36.135842 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e30ee46-58b0-4fb4-af37-bdcb932a80d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:36.136132 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e30ee46-58b0-4fb4-af37-bdcb932a80d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:36.136414 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e30ee46-58b0-4fb4-af37-bdcb932a80d2 admin admin] PATH_INFO: `/v3/projects/2bf56351e9704e01aa67d4108a915fe7` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:36.137628 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e30ee46-58b0-4fb4-af37-bdcb932a80d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2bf56351e9704e01aa67d4108a915fe7)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:13:36.139609 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e30ee46-58b0-4fb4-af37-bdcb932a80d2 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:13:36.187324 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-4e30ee46-58b0-4fb4-af37-bdcb932a80d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2bf56351e9704e01aa67d4108a915fe7', 'request_id': 'req-4e30ee46-58b0-4fb4-af37-bdcb932a80d2'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:13:36.194168 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-4e30ee46-58b0-4fb4-af37-bdcb932a80d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2bf56351e9704e01aa67d4108a915fe7', 'request_id': 'req-4e30ee46-58b0-4fb4-af37-bdcb932a80d2'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:13:36.199851 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-4e30ee46-58b0-4fb4-af37-bdcb932a80d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2bf56351e9704e01aa67d4108a915fe7', 'request_id': 'req-4e30ee46-58b0-4fb4-af37-bdcb932a80d2'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:13:36.212904 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-4e30ee46-58b0-4fb4-af37-bdcb932a80d2 admin admin] The token cache is being invalidate because project 2bf56351e9704e01aa67d4108a915fe7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 16:13:36.213196 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-4e30ee46-58b0-4fb4-af37-bdcb932a80d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:13:36.213823 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-4e30ee46-58b0-4fb4-af37-bdcb932a80d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:13:36.215365 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1296/2561] 10.222.0.99 () {58 vars in 1232 bytes} [Thu Jul 14 16:13:36 2022] DELETE /identity/v3/projects/2bf56351e9704e01aa67d4108a915fe7 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:13:42.210555 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d951b91-0f21-401f-986c-6206fad05818 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:13:42.212069 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-9d951b91-0f21-401f-986c-6206fad05818 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:13:42.224826 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d951b91-0f21-401f-986c-6206fad05818 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:13:42.281991 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddfa857-81d2-4287-b251-3cf4974c6d5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:13:42.282896 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddfa857-81d2-4287-b251-3cf4974c6d5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:42.283187 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddfa857-81d2-4287-b251-3cf4974c6d5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:42.283470 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddfa857-81d2-4287-b251-3cf4974c6d5f admin admin] PATH_INFO: `/v3/users/6f6430a3a7164db2b39c2f8b90560503` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:42.284699 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddfa857-81d2-4287-b251-3cf4974c6d5f admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f6430a3a7164db2b39c2f8b90560503)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:13:42.287079 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddfa857-81d2-4287-b251-3cf4974c6d5f admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:13:42.345512 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5ddfa857-81d2-4287-b251-3cf4974c6d5f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f6430a3a7164db2b39c2f8b90560503'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:13:42.353071 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5ddfa857-81d2-4287-b251-3cf4974c6d5f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f6430a3a7164db2b39c2f8b90560503'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:13:42.365629 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5ddfa857-81d2-4287-b251-3cf4974c6d5f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f6430a3a7164db2b39c2f8b90560503'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:13:42.366265 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5ddfa857-81d2-4287-b251-3cf4974c6d5f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f6430a3a7164db2b39c2f8b90560503'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:13:42.366870 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5ddfa857-81d2-4287-b251-3cf4974c6d5f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f6430a3a7164db2b39c2f8b90560503'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:13:42.384890 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1266/2562] 10.222.0.99 () {58 vars in 1223 bytes} [Thu Jul 14 16:13:42 2022] DELETE /identity/v3/users/6f6430a3a7164db2b39c2f8b90560503 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:13:42.389415 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b28eb6-5017-4c24-9486-40284a4a7fb4 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:42.389908 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b28eb6-5017-4c24-9486-40284a4a7fb4 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:42.390023 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b28eb6-5017-4c24-9486-40284a4a7fb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:42.408156 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-e9b28eb6-5017-4c24-9486-40284a4a7fb4 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:13:42.409371 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-e9b28eb6-5017-4c24-9486-40284a4a7fb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:13:42.471619 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1297/2563] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 16:13:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:13:42.481115 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddfa857-81d2-4287-b251-3cf4974c6d5f admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:13:42.482510 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-5ddfa857-81d2-4287-b251-3cf4974c6d5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:13:42.525345 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:13:42.542438 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddfa857-81d2-4287-b251-3cf4974c6d5f admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:13:42.546486 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0520d3-9da4-487a-ab45-7b1c6fd04e09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:13:42.547152 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0520d3-9da4-487a-ab45-7b1c6fd04e09 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:42.547471 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0520d3-9da4-487a-ab45-7b1c6fd04e09 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:42.547787 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0520d3-9da4-487a-ab45-7b1c6fd04e09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:42.557706 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0520d3-9da4-487a-ab45-7b1c6fd04e09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:13:42.559154 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0520d3-9da4-487a-ab45-7b1c6fd04e09 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:13:42.569416 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1267/2564] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:13:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:13:42.840102 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b28eb6-5017-4c24-9486-40284a4a7fb4 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:13:42.841748 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-e9b28eb6-5017-4c24-9486-40284a4a7fb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:13:42.856479 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b28eb6-5017-4c24-9486-40284a4a7fb4 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:13:42.862106 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9410ea95-c8ee-42d6-83ae-bfc14802d1f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:13:42.863059 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9410ea95-c8ee-42d6-83ae-bfc14802d1f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:42.864111 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9410ea95-c8ee-42d6-83ae-bfc14802d1f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:42.864111 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9410ea95-c8ee-42d6-83ae-bfc14802d1f5 admin admin] PATH_INFO: `/v3/projects/d5b285bdeebb4c949286a414f34e45f0` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:42.865303 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9410ea95-c8ee-42d6-83ae-bfc14802d1f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5b285bdeebb4c949286a414f34e45f0)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:13:42.867265 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9410ea95-c8ee-42d6-83ae-bfc14802d1f5 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:13:42.914651 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:13:42.916008 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-9410ea95-c8ee-42d6-83ae-bfc14802d1f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5b285bdeebb4c949286a414f34e45f0', 'request_id': 'req-9410ea95-c8ee-42d6-83ae-bfc14802d1f5'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:13:42.924893 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-9410ea95-c8ee-42d6-83ae-bfc14802d1f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5b285bdeebb4c949286a414f34e45f0', 'request_id': 'req-9410ea95-c8ee-42d6-83ae-bfc14802d1f5'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:13:42.931236 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-9410ea95-c8ee-42d6-83ae-bfc14802d1f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5b285bdeebb4c949286a414f34e45f0', 'request_id': 'req-9410ea95-c8ee-42d6-83ae-bfc14802d1f5'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:13:42.944529 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-9410ea95-c8ee-42d6-83ae-bfc14802d1f5 admin admin] The token cache is being invalidate because project d5b285bdeebb4c949286a414f34e45f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 16:13:42.944819 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-9410ea95-c8ee-42d6-83ae-bfc14802d1f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:13:42.945431 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-9410ea95-c8ee-42d6-83ae-bfc14802d1f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:13:42.946962 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1298/2565] 10.222.0.99 () {58 vars in 1232 bytes} [Thu Jul 14 16:13:42 2022] DELETE /identity/v3/projects/d5b285bdeebb4c949286a414f34e45f0 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:13:42.950963 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0520d3-9da4-487a-ab45-7b1c6fd04e09 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:13:42.961289 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0520d3-9da4-487a-ab45-7b1c6fd04e09 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:13:43.028796 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67e27e4-c180-48c3-8ac3-21c9289cac43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:13:43.029572 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67e27e4-c180-48c3-8ac3-21c9289cac43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:43.029859 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67e27e4-c180-48c3-8ac3-21c9289cac43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:43.030137 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67e27e4-c180-48c3-8ac3-21c9289cac43 admin admin] PATH_INFO: `/v3/users/8bcd44a40ec942adaf2b705000ec7b49` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:43.031553 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67e27e4-c180-48c3-8ac3-21c9289cac43 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8bcd44a40ec942adaf2b705000ec7b49)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:13:43.033837 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67e27e4-c180-48c3-8ac3-21c9289cac43 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:13:43.081399 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-c67e27e4-c180-48c3-8ac3-21c9289cac43 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8bcd44a40ec942adaf2b705000ec7b49'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:13:43.089306 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-c67e27e4-c180-48c3-8ac3-21c9289cac43 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8bcd44a40ec942adaf2b705000ec7b49'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:13:43.101233 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-c67e27e4-c180-48c3-8ac3-21c9289cac43 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8bcd44a40ec942adaf2b705000ec7b49'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:13:43.101866 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-c67e27e4-c180-48c3-8ac3-21c9289cac43 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8bcd44a40ec942adaf2b705000ec7b49'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:13:43.102483 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-c67e27e4-c180-48c3-8ac3-21c9289cac43 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8bcd44a40ec942adaf2b705000ec7b49'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:13:43.114597 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1268/2566] 10.222.0.99 () {58 vars in 1223 bytes} [Thu Jul 14 16:13:42 2022] DELETE /identity/v3/users/8bcd44a40ec942adaf2b705000ec7b49 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:13:43.396082 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9410ea95-c8ee-42d6-83ae-bfc14802d1f5 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:13:43.397757 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-9410ea95-c8ee-42d6-83ae-bfc14802d1f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:13:43.415114 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9410ea95-c8ee-42d6-83ae-bfc14802d1f5 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:13:43.477963 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be53004a-1af4-479e-95a3-d06c01af2c47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:13:43.478829 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be53004a-1af4-479e-95a3-d06c01af2c47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:43.479114 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be53004a-1af4-479e-95a3-d06c01af2c47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:43.479393 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be53004a-1af4-479e-95a3-d06c01af2c47 admin admin] PATH_INFO: `/v3/projects/6611144c16564379a5c1fb49869285ef` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:43.480614 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be53004a-1af4-479e-95a3-d06c01af2c47 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6611144c16564379a5c1fb49869285ef)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:13:43.482509 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be53004a-1af4-479e-95a3-d06c01af2c47 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:13:43.530636 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-be53004a-1af4-479e-95a3-d06c01af2c47 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6611144c16564379a5c1fb49869285ef', 'request_id': 'req-be53004a-1af4-479e-95a3-d06c01af2c47'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:13:43.537250 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-be53004a-1af4-479e-95a3-d06c01af2c47 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6611144c16564379a5c1fb49869285ef', 'request_id': 'req-be53004a-1af4-479e-95a3-d06c01af2c47'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:13:43.543105 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-be53004a-1af4-479e-95a3-d06c01af2c47 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6611144c16564379a5c1fb49869285ef', 'request_id': 'req-be53004a-1af4-479e-95a3-d06c01af2c47'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:13:43.556070 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-be53004a-1af4-479e-95a3-d06c01af2c47 admin admin] The token cache is being invalidate because project 6611144c16564379a5c1fb49869285ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 16:13:43.556356 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-be53004a-1af4-479e-95a3-d06c01af2c47 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:13:43.556984 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-be53004a-1af4-479e-95a3-d06c01af2c47 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:13:43.558530 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1299/2567] 10.222.0.99 () {58 vars in 1232 bytes} [Thu Jul 14 16:13:43 2022] DELETE /identity/v3/projects/6611144c16564379a5c1fb49869285ef => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:13:43.563505 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9e30a3-e551-4a12-a02e-b4aec1a1abf3 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:43.563986 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9e30a3-e551-4a12-a02e-b4aec1a1abf3 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:43.564112 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9e30a3-e551-4a12-a02e-b4aec1a1abf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:43.584129 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-da9e30a3-e551-4a12-a02e-b4aec1a1abf3 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:13:43.585285 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-da9e30a3-e551-4a12-a02e-b4aec1a1abf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:13:43.637806 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:13:43.647425 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1269/2568] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 16:13:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:13:43.656877 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b244705d-b630-44fb-b1e1-320bcfeefefc None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:43.657622 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b244705d-b630-44fb-b1e1-320bcfeefefc None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:43.657909 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b244705d-b630-44fb-b1e1-320bcfeefefc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:43.678362 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-b244705d-b630-44fb-b1e1-320bcfeefefc None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:13:43.679580 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-b244705d-b630-44fb-b1e1-320bcfeefefc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:13:43.687488 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1300/2569] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 16:13:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 16:13:43.691918 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9e30a3-e551-4a12-a02e-b4aec1a1abf3 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:13:43.701645 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9e30a3-e551-4a12-a02e-b4aec1a1abf3 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:13:43.706452 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356feaa6-1498-45ff-a3c0-98717263b8f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:13:43.707273 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356feaa6-1498-45ff-a3c0-98717263b8f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:43.707564 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356feaa6-1498-45ff-a3c0-98717263b8f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:43.707874 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356feaa6-1498-45ff-a3c0-98717263b8f9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:43.708627 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356feaa6-1498-45ff-a3c0-98717263b8f9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 16:13:43.708925 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356feaa6-1498-45ff-a3c0-98717263b8f9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:13:43.711036 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356feaa6-1498-45ff-a3c0-98717263b8f9 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:13:43.723088 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1270/2570] 10.222.0.99 () {58 vars in 1152 bytes} [Thu Jul 14 16:13:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 16:13:43.727654 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b244705d-b630-44fb-b1e1-320bcfeefefc None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:13:43.737468 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b244705d-b630-44fb-b1e1-320bcfeefefc None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:13:43.742823 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e3d1dd-4972-427e-bd02-1d4a38668e1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:13:43.743518 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e3d1dd-4972-427e-bd02-1d4a38668e1a admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:43.743925 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e3d1dd-4972-427e-bd02-1d4a38668e1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:43.744215 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e3d1dd-4972-427e-bd02-1d4a38668e1a admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:43.744929 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e3d1dd-4972-427e-bd02-1d4a38668e1a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:13:43.746839 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e3d1dd-4972-427e-bd02-1d4a38668e1a admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:13:43.774482 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1301/2571] 10.222.0.99 () {60 vars in 1152 bytes} [Thu Jul 14 16:13:43 2022] POST /identity/v3/projects => generated 363 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 16:13:43.778542 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356feaa6-1498-45ff-a3c0-98717263b8f9 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:13:43.789532 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356feaa6-1498-45ff-a3c0-98717263b8f9 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:13:43.845345 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd492d1-d814-4639-aaea-fd22f3adbbc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:13:43.845941 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd492d1-d814-4639-aaea-fd22f3adbbc6 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:43.846227 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd492d1-d814-4639-aaea-fd22f3adbbc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:43.846507 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd492d1-d814-4639-aaea-fd22f3adbbc6 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:43.847732 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd492d1-d814-4639-aaea-fd22f3adbbc6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:13:43.849866 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd492d1-d814-4639-aaea-fd22f3adbbc6 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:13:43.870514 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1271/2572] 10.222.0.99 () {60 vars in 1143 bytes} [Thu Jul 14 16:13:43 2022] POST /identity/v3/users => generated 282 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 16:13:43.874732 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e3d1dd-4972-427e-bd02-1d4a38668e1a admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:13:43.885295 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e3d1dd-4972-427e-bd02-1d4a38668e1a admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:13:43.889879 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee5a4b8-ff4a-4854-a003-909ad3768cd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:13:43.890536 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee5a4b8-ff4a-4854-a003-909ad3768cd8 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:43.890820 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee5a4b8-ff4a-4854-a003-909ad3768cd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:43.891097 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee5a4b8-ff4a-4854-a003-909ad3768cd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:43.891861 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee5a4b8-ff4a-4854-a003-909ad3768cd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:13:43.893452 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee5a4b8-ff4a-4854-a003-909ad3768cd8 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:13:43.903835 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1302/2573] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 16:13:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:13:43.907830 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd492d1-d814-4639-aaea-fd22f3adbbc6 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:13:43.918541 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd492d1-d814-4639-aaea-fd22f3adbbc6 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:13:43.923621 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbcdebd-1808-4b2d-83b7-9e4a9725574a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:13:43.925148 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbcdebd-1808-4b2d-83b7-9e4a9725574a admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:43.926010 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbcdebd-1808-4b2d-83b7-9e4a9725574a admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:43.926298 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbcdebd-1808-4b2d-83b7-9e4a9725574a admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:43.927117 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbcdebd-1808-4b2d-83b7-9e4a9725574a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:13:43.928800 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbcdebd-1808-4b2d-83b7-9e4a9725574a admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:13:43.939652 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1272/2574] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 16:13:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:13:43.944126 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee5a4b8-ff4a-4854-a003-909ad3768cd8 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:13:43.954198 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee5a4b8-ff4a-4854-a003-909ad3768cd8 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:13:43.959339 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994343f5-3e1e-439d-8540-0206563c5c2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:13:43.960257 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994343f5-3e1e-439d-8540-0206563c5c2c admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:43.960620 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994343f5-3e1e-439d-8540-0206563c5c2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:43.960901 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994343f5-3e1e-439d-8540-0206563c5c2c admin admin] PATH_INFO: `/v3/projects/1c149f7558264830af84facf9745f535/users/4b233611653543fc9f17d8abcfa3a21d/roles/b8c98fddc0fd4436b577fd1d3361bc38` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:43.976207 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994343f5-3e1e-439d-8540-0206563c5c2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c149f7558264830af84facf9745f535, user_id=4b233611653543fc9f17d8abcfa3a21d, role_id=b8c98fddc0fd4436b577fd1d3361bc38)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:13:43.978970 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994343f5-3e1e-439d-8540-0206563c5c2c admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:13:43.993642 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1303/2575] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 16:13:43 2022] PUT /identity/v3/projects/1c149f7558264830af84facf9745f535/users/4b233611653543fc9f17d8abcfa3a21d/roles/b8c98fddc0fd4436b577fd1d3361bc38 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 16:13:44.006017 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee77dd6-e436-48ef-9610-50fc0b107739 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:44.006318 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee77dd6-e436-48ef-9610-50fc0b107739 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:44.006605 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee77dd6-e436-48ef-9610-50fc0b107739 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:44.048168 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-8ee77dd6-e436-48ef-9610-50fc0b107739 None None] MFA Rules not processed for user `4b233611653543fc9f17d8abcfa3a21d`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:13:44.049400 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-8ee77dd6-e436-48ef-9610-50fc0b107739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:13:44.130466 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1273/2576] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 16:13:44 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:13:44.134629 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994343f5-3e1e-439d-8540-0206563c5c2c admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:13:44.139574 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:13:44.208586 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994343f5-3e1e-439d-8540-0206563c5c2c admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:13:44.213533 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9882c750-e473-48dd-a967-f99db1be9934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:13:44.214207 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9882c750-e473-48dd-a967-f99db1be9934 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:44.214665 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9882c750-e473-48dd-a967-f99db1be9934 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:44.214947 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9882c750-e473-48dd-a967-f99db1be9934 admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:44.215694 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9882c750-e473-48dd-a967-f99db1be9934 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:13:44.217612 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9882c750-e473-48dd-a967-f99db1be9934 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:13:44.244717 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1304/2577] 10.222.0.99 () {60 vars in 1152 bytes} [Thu Jul 14 16:13:44 2022] POST /identity/v3/projects => generated 365 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 16:13:44.249047 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee77dd6-e436-48ef-9610-50fc0b107739 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:13:44.260473 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee77dd6-e436-48ef-9610-50fc0b107739 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:13:44.313621 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:13:44.323275 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c1ea86-4745-469b-975f-f44b5458c901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:13:44.324013 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c1ea86-4745-469b-975f-f44b5458c901 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:44.324315 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c1ea86-4745-469b-975f-f44b5458c901 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:44.324600 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c1ea86-4745-469b-975f-f44b5458c901 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:44.325311 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c1ea86-4745-469b-975f-f44b5458c901 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:13:44.327247 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c1ea86-4745-469b-975f-f44b5458c901 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:13:44.348933 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1274/2578] 10.222.0.99 () {60 vars in 1143 bytes} [Thu Jul 14 16:13:44 2022] POST /identity/v3/users => generated 289 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 16:13:44.353011 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9882c750-e473-48dd-a967-f99db1be9934 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:13:44.363575 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9882c750-e473-48dd-a967-f99db1be9934 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:13:44.368283 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fe83b2-30ed-4510-9911-8770708eee7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:13:44.368900 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fe83b2-30ed-4510-9911-8770708eee7b admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:44.369414 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fe83b2-30ed-4510-9911-8770708eee7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:44.369414 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fe83b2-30ed-4510-9911-8770708eee7b admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:44.370200 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fe83b2-30ed-4510-9911-8770708eee7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:13:44.371930 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fe83b2-30ed-4510-9911-8770708eee7b admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:13:44.383368 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1305/2579] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 16:13:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:13:44.387570 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c1ea86-4745-469b-975f-f44b5458c901 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:13:44.398326 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c1ea86-4745-469b-975f-f44b5458c901 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:13:44.403033 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f73396-8e10-4f99-8165-6eb7046b2b07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:13:44.403960 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f73396-8e10-4f99-8165-6eb7046b2b07 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:44.404248 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f73396-8e10-4f99-8165-6eb7046b2b07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:44.404532 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f73396-8e10-4f99-8165-6eb7046b2b07 admin admin] PATH_INFO: `/v3/projects/ca4cdda0e23b4f2388c874e06640dcd2/users/fadcc79a329642c6af65da97c5760f6c/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:44.419398 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f73396-8e10-4f99-8165-6eb7046b2b07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca4cdda0e23b4f2388c874e06640dcd2, user_id=fadcc79a329642c6af65da97c5760f6c, role_id=d0f6ba1314f14d9f9c816bdb5d0c3aac)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:13:44.422399 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f73396-8e10-4f99-8165-6eb7046b2b07 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:13:44.436582 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1275/2580] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 16:13:44 2022] PUT /identity/v3/projects/ca4cdda0e23b4f2388c874e06640dcd2/users/fadcc79a329642c6af65da97c5760f6c/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 16:13:44.440607 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fe83b2-30ed-4510-9911-8770708eee7b admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:13:44.454456 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fe83b2-30ed-4510-9911-8770708eee7b admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:13:44.513900 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172b49ee-10db-47bb-b780-0f0f6857fd61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:13:44.514578 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172b49ee-10db-47bb-b780-0f0f6857fd61 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:44.514871 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172b49ee-10db-47bb-b780-0f0f6857fd61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:44.515159 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172b49ee-10db-47bb-b780-0f0f6857fd61 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:44.515928 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172b49ee-10db-47bb-b780-0f0f6857fd61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:13:44.517538 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172b49ee-10db-47bb-b780-0f0f6857fd61 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:13:44.528185 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1306/2581] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 16:13:44 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:13:44.532500 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f73396-8e10-4f99-8165-6eb7046b2b07 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:13:44.543043 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f73396-8e10-4f99-8165-6eb7046b2b07 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:13:44.548057 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892ed249-580b-427b-b0d3-da0266cb6614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:13:44.548932 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892ed249-580b-427b-b0d3-da0266cb6614 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:44.549215 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892ed249-580b-427b-b0d3-da0266cb6614 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:44.549619 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892ed249-580b-427b-b0d3-da0266cb6614 admin admin] PATH_INFO: `/v3/projects/ca4cdda0e23b4f2388c874e06640dcd2/users/fadcc79a329642c6af65da97c5760f6c/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:44.551806 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892ed249-580b-427b-b0d3-da0266cb6614 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca4cdda0e23b4f2388c874e06640dcd2, user_id=fadcc79a329642c6af65da97c5760f6c, role_id=d0f6ba1314f14d9f9c816bdb5d0c3aac)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:13:44.554533 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892ed249-580b-427b-b0d3-da0266cb6614 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:13:44.565552 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1276/2582] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 16:13:44 2022] PUT /identity/v3/projects/ca4cdda0e23b4f2388c874e06640dcd2/users/fadcc79a329642c6af65da97c5760f6c/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:13:44.578018 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a101954d-a3fb-4fc6-9bf1-7f87347bb54e None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:44.578345 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a101954d-a3fb-4fc6-9bf1-7f87347bb54e None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:44.578632 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a101954d-a3fb-4fc6-9bf1-7f87347bb54e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:44.608615 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-a101954d-a3fb-4fc6-9bf1-7f87347bb54e None None] MFA Rules not processed for user `fadcc79a329642c6af65da97c5760f6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:13:44.609866 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-a101954d-a3fb-4fc6-9bf1-7f87347bb54e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:13:44.674188 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:13:44.680823 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1307/2583] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 16:13:44 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:13:44.743180 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892ed249-580b-427b-b0d3-da0266cb6614 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:13:44.744900 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-892ed249-580b-427b-b0d3-da0266cb6614 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:13:44.802909 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892ed249-580b-427b-b0d3-da0266cb6614 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:13:44.806564 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da8aa45-3ebe-4638-9a7e-6caba710d246 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3e1a2b83f3d4d88ba5084dfaffd22f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:13:44.807211 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da8aa45-3ebe-4638-9a7e-6caba710d246 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:44.807494 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da8aa45-3ebe-4638-9a7e-6caba710d246 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:44.807809 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da8aa45-3ebe-4638-9a7e-6caba710d246 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:44.817697 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da8aa45-3ebe-4638-9a7e-6caba710d246 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:13:44.819149 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da8aa45-3ebe-4638-9a7e-6caba710d246 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:13:44.830582 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1277/2584] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:13:44 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:13:44.901522 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a101954d-a3fb-4fc6-9bf1-7f87347bb54e None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:13:44.914002 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a101954d-a3fb-4fc6-9bf1-7f87347bb54e None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:13:44.917724 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f355c8f-caa4-4618-ad03-baee3f44d166 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3e1a2b83f3d4d88ba5084dfaffd22f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:13:44.918381 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f355c8f-caa4-4618-ad03-baee3f44d166 service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:44.918671 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f355c8f-caa4-4618-ad03-baee3f44d166 service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:44.918955 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f355c8f-caa4-4618-ad03-baee3f44d166 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:44.982003 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f355c8f-caa4-4618-ad03-baee3f44d166 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:13:44.983379 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f355c8f-caa4-4618-ad03-baee3f44d166 service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:13:44.992985 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1308/2585] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:13:44 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:13:54.825639 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=45,keystone:INSERT=5 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:13:54.988881 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=29 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:13:55.188910 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da8aa45-3ebe-4638-9a7e-6caba710d246 service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:13:55.255583 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da8aa45-3ebe-4638-9a7e-6caba710d246 service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:13:55.262670 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1ce014-e861-4802-8635-5df0f8b87d03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:13:55.262670 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1ce014-e861-4802-8635-5df0f8b87d03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:55.262670 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1ce014-e861-4802-8635-5df0f8b87d03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:55.262670 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1ce014-e861-4802-8635-5df0f8b87d03 admin admin] PATH_INFO: `/v3/users/4b233611653543fc9f17d8abcfa3a21d` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:55.265679 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1ce014-e861-4802-8635-5df0f8b87d03 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b233611653543fc9f17d8abcfa3a21d)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:13:55.267631 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1ce014-e861-4802-8635-5df0f8b87d03 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:13:55.323319 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5e1ce014-e861-4802-8635-5df0f8b87d03 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b233611653543fc9f17d8abcfa3a21d'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:13:55.330810 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5e1ce014-e861-4802-8635-5df0f8b87d03 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b233611653543fc9f17d8abcfa3a21d'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:13:55.343670 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5e1ce014-e861-4802-8635-5df0f8b87d03 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b233611653543fc9f17d8abcfa3a21d'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:13:55.344326 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5e1ce014-e861-4802-8635-5df0f8b87d03 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b233611653543fc9f17d8abcfa3a21d'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:13:55.344935 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5e1ce014-e861-4802-8635-5df0f8b87d03 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b233611653543fc9f17d8abcfa3a21d'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:13:55.358503 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1278/2586] 10.222.0.99 () {58 vars in 1223 bytes} [Thu Jul 14 16:13:55 2022] DELETE /identity/v3/users/4b233611653543fc9f17d8abcfa3a21d => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:13:55.363073 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f84ea7-ae22-4665-a722-a5b5ce906d37 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:55.363578 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f84ea7-ae22-4665-a722-a5b5ce906d37 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:55.363889 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f84ea7-ae22-4665-a722-a5b5ce906d37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:55.380768 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-71f84ea7-ae22-4665-a722-a5b5ce906d37 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:13:55.381997 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-71f84ea7-ae22-4665-a722-a5b5ce906d37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:13:55.441123 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1309/2587] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 16:13:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:13:55.451289 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1ce014-e861-4802-8635-5df0f8b87d03 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:13:55.452844 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-5e1ce014-e861-4802-8635-5df0f8b87d03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:13:55.523538 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1ce014-e861-4802-8635-5df0f8b87d03 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:13:55.527116 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abf0527-8c6e-4021-86dc-5f3c75bb7191 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:13:55.527799 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abf0527-8c6e-4021-86dc-5f3c75bb7191 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:55.528097 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abf0527-8c6e-4021-86dc-5f3c75bb7191 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:55.528379 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abf0527-8c6e-4021-86dc-5f3c75bb7191 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:55.537637 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abf0527-8c6e-4021-86dc-5f3c75bb7191 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:13:55.539096 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abf0527-8c6e-4021-86dc-5f3c75bb7191 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:13:55.549912 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1279/2588] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:13:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:13:55.840585 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f84ea7-ae22-4665-a722-a5b5ce906d37 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:13:55.842137 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-71f84ea7-ae22-4665-a722-a5b5ce906d37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:13:55.856244 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f84ea7-ae22-4665-a722-a5b5ce906d37 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:13:55.861101 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2f1779-22da-4773-9835-7f58f5a12d12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:13:55.861982 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2f1779-22da-4773-9835-7f58f5a12d12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:55.862269 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2f1779-22da-4773-9835-7f58f5a12d12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:55.862551 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2f1779-22da-4773-9835-7f58f5a12d12 admin admin] PATH_INFO: `/v3/projects/1c149f7558264830af84facf9745f535` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:55.864758 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2f1779-22da-4773-9835-7f58f5a12d12 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c149f7558264830af84facf9745f535)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:13:55.867075 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2f1779-22da-4773-9835-7f58f5a12d12 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:13:55.913957 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-1f2f1779-22da-4773-9835-7f58f5a12d12 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c149f7558264830af84facf9745f535', 'request_id': 'req-1f2f1779-22da-4773-9835-7f58f5a12d12'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:13:55.919998 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-1f2f1779-22da-4773-9835-7f58f5a12d12 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c149f7558264830af84facf9745f535', 'request_id': 'req-1f2f1779-22da-4773-9835-7f58f5a12d12'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:13:55.926556 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-1f2f1779-22da-4773-9835-7f58f5a12d12 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c149f7558264830af84facf9745f535', 'request_id': 'req-1f2f1779-22da-4773-9835-7f58f5a12d12'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:13:55.938349 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-1f2f1779-22da-4773-9835-7f58f5a12d12 admin admin] The token cache is being invalidate because project 1c149f7558264830af84facf9745f535 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 16:13:55.938641 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-1f2f1779-22da-4773-9835-7f58f5a12d12 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:13:55.939246 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-1f2f1779-22da-4773-9835-7f58f5a12d12 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:13:55.940855 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1310/2589] 10.222.0.99 () {58 vars in 1232 bytes} [Thu Jul 14 16:13:55 2022] DELETE /identity/v3/projects/1c149f7558264830af84facf9745f535 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:13:55.945109 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abf0527-8c6e-4021-86dc-5f3c75bb7191 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:13:55.954995 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abf0527-8c6e-4021-86dc-5f3c75bb7191 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:13:55.963731 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:13:56.018595 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d75e53e-77e0-4ace-8f48-dfd63cde320d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:13:56.020020 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d75e53e-77e0-4ace-8f48-dfd63cde320d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:56.020307 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d75e53e-77e0-4ace-8f48-dfd63cde320d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:56.020584 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d75e53e-77e0-4ace-8f48-dfd63cde320d admin admin] PATH_INFO: `/v3/users/fadcc79a329642c6af65da97c5760f6c` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:56.021782 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d75e53e-77e0-4ace-8f48-dfd63cde320d admin admin] RBAC: Authorizing `identity:delete_user(user_id=fadcc79a329642c6af65da97c5760f6c)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:13:56.023886 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d75e53e-77e0-4ace-8f48-dfd63cde320d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:13:56.071399 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-7d75e53e-77e0-4ace-8f48-dfd63cde320d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fadcc79a329642c6af65da97c5760f6c'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:13:56.077930 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-7d75e53e-77e0-4ace-8f48-dfd63cde320d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fadcc79a329642c6af65da97c5760f6c'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:13:56.090642 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-7d75e53e-77e0-4ace-8f48-dfd63cde320d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fadcc79a329642c6af65da97c5760f6c'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:13:56.091298 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-7d75e53e-77e0-4ace-8f48-dfd63cde320d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fadcc79a329642c6af65da97c5760f6c'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:13:56.091943 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-7d75e53e-77e0-4ace-8f48-dfd63cde320d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fadcc79a329642c6af65da97c5760f6c'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:13:56.104849 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1280/2590] 10.222.0.99 () {58 vars in 1223 bytes} [Thu Jul 14 16:13:55 2022] DELETE /identity/v3/users/fadcc79a329642c6af65da97c5760f6c => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:13:56.380182 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2f1779-22da-4773-9835-7f58f5a12d12 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:13:56.381678 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-1f2f1779-22da-4773-9835-7f58f5a12d12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:13:56.396302 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2f1779-22da-4773-9835-7f58f5a12d12 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:13:56.454451 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37e30da-90bb-4d96-b447-ea5101da6dde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:13:56.455344 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37e30da-90bb-4d96-b447-ea5101da6dde admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:56.455660 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37e30da-90bb-4d96-b447-ea5101da6dde admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:56.455947 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37e30da-90bb-4d96-b447-ea5101da6dde admin admin] PATH_INFO: `/v3/projects/ca4cdda0e23b4f2388c874e06640dcd2` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:56.457263 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37e30da-90bb-4d96-b447-ea5101da6dde admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca4cdda0e23b4f2388c874e06640dcd2)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:13:56.459222 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37e30da-90bb-4d96-b447-ea5101da6dde admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:13:56.492603 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:13:56.510894 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-e37e30da-90bb-4d96-b447-ea5101da6dde admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca4cdda0e23b4f2388c874e06640dcd2', 'request_id': 'req-e37e30da-90bb-4d96-b447-ea5101da6dde'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:13:56.517078 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-e37e30da-90bb-4d96-b447-ea5101da6dde admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca4cdda0e23b4f2388c874e06640dcd2', 'request_id': 'req-e37e30da-90bb-4d96-b447-ea5101da6dde'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:13:56.522715 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-e37e30da-90bb-4d96-b447-ea5101da6dde admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca4cdda0e23b4f2388c874e06640dcd2', 'request_id': 'req-e37e30da-90bb-4d96-b447-ea5101da6dde'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:13:56.536124 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-e37e30da-90bb-4d96-b447-ea5101da6dde admin admin] The token cache is being invalidate because project ca4cdda0e23b4f2388c874e06640dcd2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 16:13:56.536414 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-e37e30da-90bb-4d96-b447-ea5101da6dde admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:13:56.537038 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-e37e30da-90bb-4d96-b447-ea5101da6dde admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:13:56.538675 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1311/2591] 10.222.0.99 () {58 vars in 1232 bytes} [Thu Jul 14 16:13:56 2022] DELETE /identity/v3/projects/ca4cdda0e23b4f2388c874e06640dcd2 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 16:13:56.543648 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bea61fe-56ac-4aab-93a2-4f7f85874b5b None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:56.543945 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bea61fe-56ac-4aab-93a2-4f7f85874b5b None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:56.544227 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bea61fe-56ac-4aab-93a2-4f7f85874b5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:56.564280 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-3bea61fe-56ac-4aab-93a2-4f7f85874b5b None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:13:56.565498 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-3bea61fe-56ac-4aab-93a2-4f7f85874b5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:13:56.628172 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1281/2592] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 16:13:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:13:56.636589 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9278e486-1d3f-48f7-97bb-85047dbff6ea None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:56.636943 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9278e486-1d3f-48f7-97bb-85047dbff6ea None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:56.637228 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9278e486-1d3f-48f7-97bb-85047dbff6ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:56.655480 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-9278e486-1d3f-48f7-97bb-85047dbff6ea None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:13:56.656711 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-9278e486-1d3f-48f7-97bb-85047dbff6ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:13:56.665599 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1312/2593] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 16:13:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:13:56.670029 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bea61fe-56ac-4aab-93a2-4f7f85874b5b None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:13:56.681783 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bea61fe-56ac-4aab-93a2-4f7f85874b5b None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:13:56.686387 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975c5657-bc62-40df-8230-a3a59b1220c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:13:56.687052 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975c5657-bc62-40df-8230-a3a59b1220c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:56.687344 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975c5657-bc62-40df-8230-a3a59b1220c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:56.687683 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975c5657-bc62-40df-8230-a3a59b1220c3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:56.688417 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975c5657-bc62-40df-8230-a3a59b1220c3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 16:13:56.688731 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975c5657-bc62-40df-8230-a3a59b1220c3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:13:56.690374 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975c5657-bc62-40df-8230-a3a59b1220c3 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:13:56.702495 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1282/2594] 10.222.0.99 () {58 vars in 1152 bytes} [Thu Jul 14 16:13:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 16:13:56.707522 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9278e486-1d3f-48f7-97bb-85047dbff6ea None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:13:56.717093 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9278e486-1d3f-48f7-97bb-85047dbff6ea None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:13:56.721738 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfad3a7-b4ca-42ee-a85b-c2b0984527a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:13:56.723030 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfad3a7-b4ca-42ee-a85b-c2b0984527a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:56.723313 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfad3a7-b4ca-42ee-a85b-c2b0984527a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:56.723764 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfad3a7-b4ca-42ee-a85b-c2b0984527a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:56.724318 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfad3a7-b4ca-42ee-a85b-c2b0984527a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:13:56.726219 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfad3a7-b4ca-42ee-a85b-c2b0984527a1 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:13:56.751391 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1313/2595] 10.222.0.99 () {60 vars in 1152 bytes} [Thu Jul 14 16:13:56 2022] POST /identity/v3/projects => generated 377 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 16:13:56.755713 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975c5657-bc62-40df-8230-a3a59b1220c3 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:13:56.765975 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975c5657-bc62-40df-8230-a3a59b1220c3 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:13:56.778908 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:13:56.825832 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b008ee3f-f3e7-4d60-bf02-b36e5454ccc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:13:56.826519 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b008ee3f-f3e7-4d60-bf02-b36e5454ccc7 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:56.826808 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b008ee3f-f3e7-4d60-bf02-b36e5454ccc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:56.827092 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b008ee3f-f3e7-4d60-bf02-b36e5454ccc7 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:56.827872 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b008ee3f-f3e7-4d60-bf02-b36e5454ccc7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:13:56.829784 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b008ee3f-f3e7-4d60-bf02-b36e5454ccc7 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:13:56.854850 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1283/2596] 10.222.0.99 () {60 vars in 1143 bytes} [Thu Jul 14 16:13:56 2022] POST /identity/v3/users => generated 289 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 16:13:56.856104 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfad3a7-b4ca-42ee-a85b-c2b0984527a1 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:13:56.866777 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfad3a7-b4ca-42ee-a85b-c2b0984527a1 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:13:56.871444 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e457536-bc9b-480c-9af2-696f4b239722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:13:56.872139 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e457536-bc9b-480c-9af2-696f4b239722 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:56.872430 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e457536-bc9b-480c-9af2-696f4b239722 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:56.872714 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e457536-bc9b-480c-9af2-696f4b239722 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:56.873447 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e457536-bc9b-480c-9af2-696f4b239722 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:13:56.875042 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e457536-bc9b-480c-9af2-696f4b239722 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:13:56.885609 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1314/2597] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 16:13:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:13:56.889463 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b008ee3f-f3e7-4d60-bf02-b36e5454ccc7 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:13:56.900265 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b008ee3f-f3e7-4d60-bf02-b36e5454ccc7 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:13:56.904950 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03476d2-7a3d-4230-aeb8-3a23913f179a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:13:56.905636 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03476d2-7a3d-4230-aeb8-3a23913f179a admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:56.905929 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03476d2-7a3d-4230-aeb8-3a23913f179a admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:56.906213 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03476d2-7a3d-4230-aeb8-3a23913f179a admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:56.906943 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03476d2-7a3d-4230-aeb8-3a23913f179a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:13:56.908574 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03476d2-7a3d-4230-aeb8-3a23913f179a admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:13:56.918770 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1284/2598] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 16:13:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:13:56.923036 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e457536-bc9b-480c-9af2-696f4b239722 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:13:56.935172 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e457536-bc9b-480c-9af2-696f4b239722 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:13:56.940010 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb441636-a9fe-4ba1-bb63-6115b7b0bb6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:13:56.941852 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb441636-a9fe-4ba1-bb63-6115b7b0bb6d admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:56.942116 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb441636-a9fe-4ba1-bb63-6115b7b0bb6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:56.942643 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb441636-a9fe-4ba1-bb63-6115b7b0bb6d admin admin] PATH_INFO: `/v3/projects/9643505587fa48caa5545018d8e04ee8/users/3f8e17e991ed4251939ff57c308c9937/roles/b8c98fddc0fd4436b577fd1d3361bc38` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:56.958556 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb441636-a9fe-4ba1-bb63-6115b7b0bb6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9643505587fa48caa5545018d8e04ee8, user_id=3f8e17e991ed4251939ff57c308c9937, role_id=b8c98fddc0fd4436b577fd1d3361bc38)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:13:56.961243 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb441636-a9fe-4ba1-bb63-6115b7b0bb6d admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:13:56.975015 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1315/2599] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 16:13:56 2022] PUT /identity/v3/projects/9643505587fa48caa5545018d8e04ee8/users/3f8e17e991ed4251939ff57c308c9937/roles/b8c98fddc0fd4436b577fd1d3361bc38 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 16:13:56.985502 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edc0108-550a-4cd3-a2a1-f5c6eb924b61 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:56.985989 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edc0108-550a-4cd3-a2a1-f5c6eb924b61 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:56.986197 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edc0108-550a-4cd3-a2a1-f5c6eb924b61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:57.018826 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-2edc0108-550a-4cd3-a2a1-f5c6eb924b61 None None] MFA Rules not processed for user `3f8e17e991ed4251939ff57c308c9937`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:13:57.020694 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-2edc0108-550a-4cd3-a2a1-f5c6eb924b61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:13:57.086351 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1285/2600] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 16:13:56 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 16:13:57.090902 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb441636-a9fe-4ba1-bb63-6115b7b0bb6d admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:13:57.152945 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb441636-a9fe-4ba1-bb63-6115b7b0bb6d admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:13:57.157737 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934cb4cf-ed13-4ccc-bd3a-9b651fdf4c9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:13:57.158399 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934cb4cf-ed13-4ccc-bd3a-9b651fdf4c9a admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:57.158690 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934cb4cf-ed13-4ccc-bd3a-9b651fdf4c9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:57.158978 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934cb4cf-ed13-4ccc-bd3a-9b651fdf4c9a admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:57.160526 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934cb4cf-ed13-4ccc-bd3a-9b651fdf4c9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:13:57.162403 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934cb4cf-ed13-4ccc-bd3a-9b651fdf4c9a admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:13:57.188144 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1316/2601] 10.222.0.99 () {60 vars in 1152 bytes} [Thu Jul 14 16:13:57 2022] POST /identity/v3/projects => generated 379 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 16:13:57.192767 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edc0108-550a-4cd3-a2a1-f5c6eb924b61 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:13:57.203466 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edc0108-550a-4cd3-a2a1-f5c6eb924b61 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:13:57.259527 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1780cce9-3359-47e6-99e0-76279257c9ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:13:57.260188 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1780cce9-3359-47e6-99e0-76279257c9ab admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:57.260476 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1780cce9-3359-47e6-99e0-76279257c9ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:57.260761 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1780cce9-3359-47e6-99e0-76279257c9ab admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:57.261450 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1780cce9-3359-47e6-99e0-76279257c9ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:13:57.263416 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1780cce9-3359-47e6-99e0-76279257c9ab admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:13:57.283157 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1286/2602] 10.222.0.99 () {60 vars in 1143 bytes} [Thu Jul 14 16:13:57 2022] POST /identity/v3/users => generated 290 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 16:13:57.287393 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934cb4cf-ed13-4ccc-bd3a-9b651fdf4c9a admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:13:57.297971 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934cb4cf-ed13-4ccc-bd3a-9b651fdf4c9a admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:13:57.303282 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03db570-8a31-48e4-8735-b9f6d7f0123a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:13:57.304324 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03db570-8a31-48e4-8735-b9f6d7f0123a admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:57.304827 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03db570-8a31-48e4-8735-b9f6d7f0123a admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:57.305034 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03db570-8a31-48e4-8735-b9f6d7f0123a admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:57.305783 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03db570-8a31-48e4-8735-b9f6d7f0123a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:13:57.307402 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03db570-8a31-48e4-8735-b9f6d7f0123a admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:13:57.319613 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1317/2603] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 16:13:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:13:57.323803 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1780cce9-3359-47e6-99e0-76279257c9ab admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:13:57.334537 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1780cce9-3359-47e6-99e0-76279257c9ab admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:13:57.339194 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48c5118-4553-4640-8831-3725bd6efc2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:13:57.339883 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48c5118-4553-4640-8831-3725bd6efc2b admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:57.340166 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48c5118-4553-4640-8831-3725bd6efc2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:57.340443 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48c5118-4553-4640-8831-3725bd6efc2b admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:57.341175 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48c5118-4553-4640-8831-3725bd6efc2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:13:57.342772 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48c5118-4553-4640-8831-3725bd6efc2b admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:13:57.352901 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1287/2604] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 16:13:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:13:57.357133 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03db570-8a31-48e4-8735-b9f6d7f0123a admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:13:57.367251 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03db570-8a31-48e4-8735-b9f6d7f0123a admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:13:57.371920 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928ecc38-e897-4b29-a188-39d0fb1f27b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:13:57.372831 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928ecc38-e897-4b29-a188-39d0fb1f27b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:57.373094 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928ecc38-e897-4b29-a188-39d0fb1f27b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:57.373369 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928ecc38-e897-4b29-a188-39d0fb1f27b2 admin admin] PATH_INFO: `/v3/projects/dd4059a01a2a4410a0ef796729f9f144/users/f1d8a3ee7fda476ca816122199646f00/roles/b8c98fddc0fd4436b577fd1d3361bc38` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:57.388539 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928ecc38-e897-4b29-a188-39d0fb1f27b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd4059a01a2a4410a0ef796729f9f144, user_id=f1d8a3ee7fda476ca816122199646f00, role_id=b8c98fddc0fd4436b577fd1d3361bc38)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:13:57.391262 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928ecc38-e897-4b29-a188-39d0fb1f27b2 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:13:57.400764 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:DELETE=6,keystone:SELECT=89,keystone:INSERT=5 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:13:57.406213 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1318/2605] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 16:13:57 2022] PUT /identity/v3/projects/dd4059a01a2a4410a0ef796729f9f144/users/f1d8a3ee7fda476ca816122199646f00/roles/b8c98fddc0fd4436b577fd1d3361bc38 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:13:57.417746 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfba2423-08fa-4c07-9318-2d373dcc6497 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:57.418037 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfba2423-08fa-4c07-9318-2d373dcc6497 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:57.418317 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfba2423-08fa-4c07-9318-2d373dcc6497 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:57.425275 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:13:57.446980 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-bfba2423-08fa-4c07-9318-2d373dcc6497 None None] MFA Rules not processed for user `f1d8a3ee7fda476ca816122199646f00`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:13:57.448211 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-bfba2423-08fa-4c07-9318-2d373dcc6497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:13:57.514612 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1288/2606] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 16:13:57 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:13:57.519213 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928ecc38-e897-4b29-a188-39d0fb1f27b2 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:13:57.583788 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928ecc38-e897-4b29-a188-39d0fb1f27b2 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:13:57.588632 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebc13ab-d6d2-46b7-b49a-4cb0bb6ab567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:13:57.589298 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebc13ab-d6d2-46b7-b49a-4cb0bb6ab567 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:57.589597 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebc13ab-d6d2-46b7-b49a-4cb0bb6ab567 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:57.589873 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebc13ab-d6d2-46b7-b49a-4cb0bb6ab567 admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:57.590569 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebc13ab-d6d2-46b7-b49a-4cb0bb6ab567 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:13:57.592530 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebc13ab-d6d2-46b7-b49a-4cb0bb6ab567 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:13:57.625268 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1319/2607] 10.222.0.99 () {60 vars in 1152 bytes} [Thu Jul 14 16:13:57 2022] POST /identity/v3/projects => generated 379 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 16:13:57.626802 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfba2423-08fa-4c07-9318-2d373dcc6497 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:13:57.638504 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfba2423-08fa-4c07-9318-2d373dcc6497 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:13:57.698440 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688326a7-a81b-426c-bdcb-1562a1c29130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:13:57.699199 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688326a7-a81b-426c-bdcb-1562a1c29130 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:57.699537 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688326a7-a81b-426c-bdcb-1562a1c29130 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:57.699922 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688326a7-a81b-426c-bdcb-1562a1c29130 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:57.700710 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688326a7-a81b-426c-bdcb-1562a1c29130 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:13:57.702807 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688326a7-a81b-426c-bdcb-1562a1c29130 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:13:57.722647 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1289/2608] 10.222.0.99 () {60 vars in 1143 bytes} [Thu Jul 14 16:13:57 2022] POST /identity/v3/users => generated 296 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 16:13:57.726705 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebc13ab-d6d2-46b7-b49a-4cb0bb6ab567 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:13:57.737076 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebc13ab-d6d2-46b7-b49a-4cb0bb6ab567 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:13:57.742789 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce770a3c-1a3a-4eba-9aa0-234da3d86911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:13:57.743454 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce770a3c-1a3a-4eba-9aa0-234da3d86911 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:57.743785 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce770a3c-1a3a-4eba-9aa0-234da3d86911 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:57.744069 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce770a3c-1a3a-4eba-9aa0-234da3d86911 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:57.744803 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce770a3c-1a3a-4eba-9aa0-234da3d86911 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:13:57.746431 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce770a3c-1a3a-4eba-9aa0-234da3d86911 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:13:57.759460 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1320/2609] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 16:13:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:13:57.763738 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688326a7-a81b-426c-bdcb-1562a1c29130 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:13:57.774014 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688326a7-a81b-426c-bdcb-1562a1c29130 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:13:57.778833 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21388c48-0897-438f-8efb-ae271778591f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:13:57.779748 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21388c48-0897-438f-8efb-ae271778591f admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:57.780044 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21388c48-0897-438f-8efb-ae271778591f admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:57.780327 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21388c48-0897-438f-8efb-ae271778591f admin admin] PATH_INFO: `/v3/projects/4504d7ba218f43bf93d535ad1b9aa8f0/users/248768777c88479a948983e81b5ccad4/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:57.795458 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21388c48-0897-438f-8efb-ae271778591f admin admin] RBAC: Authorizing `identity:create_grant(project_id=4504d7ba218f43bf93d535ad1b9aa8f0, user_id=248768777c88479a948983e81b5ccad4, role_id=d0f6ba1314f14d9f9c816bdb5d0c3aac)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:13:57.798521 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21388c48-0897-438f-8efb-ae271778591f admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:13:57.812817 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1290/2610] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 16:13:57 2022] PUT /identity/v3/projects/4504d7ba218f43bf93d535ad1b9aa8f0/users/248768777c88479a948983e81b5ccad4/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:13:57.817437 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce770a3c-1a3a-4eba-9aa0-234da3d86911 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:13:57.828082 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce770a3c-1a3a-4eba-9aa0-234da3d86911 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:13:57.888086 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5e226a-55f2-4340-b379-33bc1116cc73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:13:57.888798 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5e226a-55f2-4340-b379-33bc1116cc73 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:57.889087 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5e226a-55f2-4340-b379-33bc1116cc73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:57.889489 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5e226a-55f2-4340-b379-33bc1116cc73 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:57.890231 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5e226a-55f2-4340-b379-33bc1116cc73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:13:57.891885 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5e226a-55f2-4340-b379-33bc1116cc73 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:13:57.902487 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1321/2611] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 16:13:57 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:13:57.906409 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21388c48-0897-438f-8efb-ae271778591f admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:13:57.917469 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21388c48-0897-438f-8efb-ae271778591f admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:13:57.922417 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ddca5d-8d4c-44d7-9b5c-0d9d180b52c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:13:57.923357 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ddca5d-8d4c-44d7-9b5c-0d9d180b52c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:57.923619 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ddca5d-8d4c-44d7-9b5c-0d9d180b52c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:57.923912 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ddca5d-8d4c-44d7-9b5c-0d9d180b52c0 admin admin] PATH_INFO: `/v3/projects/4504d7ba218f43bf93d535ad1b9aa8f0/users/248768777c88479a948983e81b5ccad4/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:57.926625 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ddca5d-8d4c-44d7-9b5c-0d9d180b52c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4504d7ba218f43bf93d535ad1b9aa8f0, user_id=248768777c88479a948983e81b5ccad4, role_id=d0f6ba1314f14d9f9c816bdb5d0c3aac)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:13:57.929712 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ddca5d-8d4c-44d7-9b5c-0d9d180b52c0 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:13:57.940110 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1291/2612] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 16:13:57 2022] PUT /identity/v3/projects/4504d7ba218f43bf93d535ad1b9aa8f0/users/248768777c88479a948983e81b5ccad4/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:13:57.950427 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70034db0-670d-4af6-9f6f-4d0de936bda8 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:57.950724 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70034db0-670d-4af6-9f6f-4d0de936bda8 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:57.951009 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70034db0-670d-4af6-9f6f-4d0de936bda8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:57.979496 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-70034db0-670d-4af6-9f6f-4d0de936bda8 None None] MFA Rules not processed for user `248768777c88479a948983e81b5ccad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:13:57.980802 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-70034db0-670d-4af6-9f6f-4d0de936bda8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:13:58.042267 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:13:58.053633 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1322/2613] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 16:13:57 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:13:58.066811 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ddca5d-8d4c-44d7-9b5c-0d9d180b52c0 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:13:58.068655 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-28ddca5d-8d4c-44d7-9b5c-0d9d180b52c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:13:58.122148 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:13:58.128023 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ddca5d-8d4c-44d7-9b5c-0d9d180b52c0 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:13:58.132672 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4264c711-c168-494e-aecd-2d6e26528272 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3e1a2b83f3d4d88ba5084dfaffd22f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:13:58.133320 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4264c711-c168-494e-aecd-2d6e26528272 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:58.133610 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4264c711-c168-494e-aecd-2d6e26528272 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:58.133888 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4264c711-c168-494e-aecd-2d6e26528272 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:58.142827 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4264c711-c168-494e-aecd-2d6e26528272 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:13:58.144759 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4264c711-c168-494e-aecd-2d6e26528272 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:13:58.154349 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1292/2614] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:13:58 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:13:58.194597 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70034db0-670d-4af6-9f6f-4d0de936bda8 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:13:58.196577 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-70034db0-670d-4af6-9f6f-4d0de936bda8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:13:58.209427 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70034db0-670d-4af6-9f6f-4d0de936bda8 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:13:58.213163 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa10dbc6-dfa3-4d21-8d28-a2dcfd531977 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3e1a2b83f3d4d88ba5084dfaffd22f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:13:58.213804 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa10dbc6-dfa3-4d21-8d28-a2dcfd531977 service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:13:58.214088 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa10dbc6-dfa3-4d21-8d28-a2dcfd531977 service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:13:58.214366 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa10dbc6-dfa3-4d21-8d28-a2dcfd531977 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:13:58.271024 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa10dbc6-dfa3-4d21-8d28-a2dcfd531977 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:13:58.272528 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa10dbc6-dfa3-4d21-8d28-a2dcfd531977 service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:13:58.281527 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1323/2615] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:13:58 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:14:01.146310 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4264c711-c168-494e-aecd-2d6e26528272 service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:14:01.209477 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4264c711-c168-494e-aecd-2d6e26528272 service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:14:01.214037 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403fcd2d-47c4-4daa-adfc-3ed5cc4912ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:14:01.214903 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403fcd2d-47c4-4daa-adfc-3ed5cc4912ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:01.215190 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403fcd2d-47c4-4daa-adfc-3ed5cc4912ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:01.215470 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403fcd2d-47c4-4daa-adfc-3ed5cc4912ba admin admin] PATH_INFO: `/v3/users/3f8e17e991ed4251939ff57c308c9937` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:01.216683 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403fcd2d-47c4-4daa-adfc-3ed5cc4912ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f8e17e991ed4251939ff57c308c9937)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:14:01.218609 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403fcd2d-47c4-4daa-adfc-3ed5cc4912ba admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:14:01.266031 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-403fcd2d-47c4-4daa-adfc-3ed5cc4912ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f8e17e991ed4251939ff57c308c9937'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:01.273381 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-403fcd2d-47c4-4daa-adfc-3ed5cc4912ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f8e17e991ed4251939ff57c308c9937'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:01.285830 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-403fcd2d-47c4-4daa-adfc-3ed5cc4912ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f8e17e991ed4251939ff57c308c9937'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:01.286491 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-403fcd2d-47c4-4daa-adfc-3ed5cc4912ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f8e17e991ed4251939ff57c308c9937'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:01.287083 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-403fcd2d-47c4-4daa-adfc-3ed5cc4912ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f8e17e991ed4251939ff57c308c9937'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:01.299405 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1293/2616] 10.222.0.99 () {58 vars in 1223 bytes} [Thu Jul 14 16:14:01 2022] DELETE /identity/v3/users/3f8e17e991ed4251939ff57c308c9937 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:14:01.304176 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcb7e05-8655-463b-b5f2-9174023629ec None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:01.304665 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcb7e05-8655-463b-b5f2-9174023629ec None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:01.304779 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcb7e05-8655-463b-b5f2-9174023629ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:01.321405 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-0bcb7e05-8655-463b-b5f2-9174023629ec None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:14:01.322618 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-0bcb7e05-8655-463b-b5f2-9174023629ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:14:01.381443 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1324/2617] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 16:14:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:14:01.390975 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403fcd2d-47c4-4daa-adfc-3ed5cc4912ba admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:14:01.392433 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-403fcd2d-47c4-4daa-adfc-3ed5cc4912ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:14:01.452032 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403fcd2d-47c4-4daa-adfc-3ed5cc4912ba admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:14:01.455712 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a4ecf9-7001-4c22-8b38-1a1cbd7de45f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:14:01.456910 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a4ecf9-7001-4c22-8b38-1a1cbd7de45f service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:01.457423 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a4ecf9-7001-4c22-8b38-1a1cbd7de45f service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:01.457946 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a4ecf9-7001-4c22-8b38-1a1cbd7de45f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:01.468224 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a4ecf9-7001-4c22-8b38-1a1cbd7de45f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:14:01.469651 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a4ecf9-7001-4c22-8b38-1a1cbd7de45f service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:14:01.480571 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1294/2618] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:14:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:14:01.757044 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcb7e05-8655-463b-b5f2-9174023629ec None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:14:01.758496 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-0bcb7e05-8655-463b-b5f2-9174023629ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:14:01.774205 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcb7e05-8655-463b-b5f2-9174023629ec None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:14:01.779080 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36fa60d-9f8c-4d89-9a19-0452bae3afb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:14:01.780018 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36fa60d-9f8c-4d89-9a19-0452bae3afb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:01.780787 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36fa60d-9f8c-4d89-9a19-0452bae3afb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:01.780787 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36fa60d-9f8c-4d89-9a19-0452bae3afb3 admin admin] PATH_INFO: `/v3/projects/9643505587fa48caa5545018d8e04ee8` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:01.781761 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36fa60d-9f8c-4d89-9a19-0452bae3afb3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9643505587fa48caa5545018d8e04ee8)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:14:01.783696 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36fa60d-9f8c-4d89-9a19-0452bae3afb3 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:14:01.832285 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-d36fa60d-9f8c-4d89-9a19-0452bae3afb3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9643505587fa48caa5545018d8e04ee8', 'request_id': 'req-d36fa60d-9f8c-4d89-9a19-0452bae3afb3'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:01.838535 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-d36fa60d-9f8c-4d89-9a19-0452bae3afb3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9643505587fa48caa5545018d8e04ee8', 'request_id': 'req-d36fa60d-9f8c-4d89-9a19-0452bae3afb3'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:01.844063 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-d36fa60d-9f8c-4d89-9a19-0452bae3afb3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9643505587fa48caa5545018d8e04ee8', 'request_id': 'req-d36fa60d-9f8c-4d89-9a19-0452bae3afb3'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:01.856738 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-d36fa60d-9f8c-4d89-9a19-0452bae3afb3 admin admin] The token cache is being invalidate because project 9643505587fa48caa5545018d8e04ee8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 16:14:01.857047 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-d36fa60d-9f8c-4d89-9a19-0452bae3afb3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:01.857818 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-d36fa60d-9f8c-4d89-9a19-0452bae3afb3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:01.859497 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1325/2619] 10.222.0.99 () {58 vars in 1232 bytes} [Thu Jul 14 16:14:01 2022] DELETE /identity/v3/projects/9643505587fa48caa5545018d8e04ee8 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 16:14:01.863898 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a4ecf9-7001-4c22-8b38-1a1cbd7de45f service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:14:01.873718 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a4ecf9-7001-4c22-8b38-1a1cbd7de45f service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:14:01.882224 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:14:01.932554 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cef3db-b1f0-44e5-b761-85748d34bda0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:14:01.933460 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cef3db-b1f0-44e5-b761-85748d34bda0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:01.933748 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cef3db-b1f0-44e5-b761-85748d34bda0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:01.934027 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cef3db-b1f0-44e5-b761-85748d34bda0 admin admin] PATH_INFO: `/v3/users/f1d8a3ee7fda476ca816122199646f00` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:01.935239 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cef3db-b1f0-44e5-b761-85748d34bda0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1d8a3ee7fda476ca816122199646f00)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:14:01.937226 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cef3db-b1f0-44e5-b761-85748d34bda0 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:14:01.985325 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d6cef3db-b1f0-44e5-b761-85748d34bda0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1d8a3ee7fda476ca816122199646f00'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:01.994745 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d6cef3db-b1f0-44e5-b761-85748d34bda0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1d8a3ee7fda476ca816122199646f00'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:02.008247 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d6cef3db-b1f0-44e5-b761-85748d34bda0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1d8a3ee7fda476ca816122199646f00'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:02.008914 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d6cef3db-b1f0-44e5-b761-85748d34bda0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1d8a3ee7fda476ca816122199646f00'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:02.009515 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d6cef3db-b1f0-44e5-b761-85748d34bda0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1d8a3ee7fda476ca816122199646f00'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:02.021653 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1295/2620] 10.222.0.99 () {58 vars in 1223 bytes} [Thu Jul 14 16:14:01 2022] DELETE /identity/v3/users/f1d8a3ee7fda476ca816122199646f00 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:14:02.299682 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36fa60d-9f8c-4d89-9a19-0452bae3afb3 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:14:02.301722 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-d36fa60d-9f8c-4d89-9a19-0452bae3afb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:14:02.317290 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36fa60d-9f8c-4d89-9a19-0452bae3afb3 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:14:02.331533 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:14:02.378804 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75088371-eede-49df-a2cc-6eb7d001cad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:14:02.379647 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75088371-eede-49df-a2cc-6eb7d001cad4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:02.379938 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75088371-eede-49df-a2cc-6eb7d001cad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:02.380222 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75088371-eede-49df-a2cc-6eb7d001cad4 admin admin] PATH_INFO: `/v3/projects/dd4059a01a2a4410a0ef796729f9f144` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:02.381526 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75088371-eede-49df-a2cc-6eb7d001cad4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd4059a01a2a4410a0ef796729f9f144)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:14:02.383496 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75088371-eede-49df-a2cc-6eb7d001cad4 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:14:02.430641 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-75088371-eede-49df-a2cc-6eb7d001cad4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd4059a01a2a4410a0ef796729f9f144', 'request_id': 'req-75088371-eede-49df-a2cc-6eb7d001cad4'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:02.436714 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-75088371-eede-49df-a2cc-6eb7d001cad4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd4059a01a2a4410a0ef796729f9f144', 'request_id': 'req-75088371-eede-49df-a2cc-6eb7d001cad4'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:02.442532 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-75088371-eede-49df-a2cc-6eb7d001cad4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd4059a01a2a4410a0ef796729f9f144', 'request_id': 'req-75088371-eede-49df-a2cc-6eb7d001cad4'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:02.455230 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-75088371-eede-49df-a2cc-6eb7d001cad4 admin admin] The token cache is being invalidate because project dd4059a01a2a4410a0ef796729f9f144 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 16:14:02.455566 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-75088371-eede-49df-a2cc-6eb7d001cad4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:02.456221 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-75088371-eede-49df-a2cc-6eb7d001cad4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:02.458350 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1326/2621] 10.222.0.99 () {58 vars in 1232 bytes} [Thu Jul 14 16:14:02 2022] DELETE /identity/v3/projects/dd4059a01a2a4410a0ef796729f9f144 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:14:02.462339 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cef3db-b1f0-44e5-b761-85748d34bda0 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:14:02.463868 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d6cef3db-b1f0-44e5-b761-85748d34bda0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:14:02.477229 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cef3db-b1f0-44e5-b761-85748d34bda0 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:14:02.533561 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e138f4d9-4a1e-4ab1-ae40-72ecb6cdd124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:14:02.535042 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e138f4d9-4a1e-4ab1-ae40-72ecb6cdd124 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:02.535042 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e138f4d9-4a1e-4ab1-ae40-72ecb6cdd124 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:02.535663 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e138f4d9-4a1e-4ab1-ae40-72ecb6cdd124 admin admin] PATH_INFO: `/v3/users/248768777c88479a948983e81b5ccad4` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:02.536936 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e138f4d9-4a1e-4ab1-ae40-72ecb6cdd124 admin admin] RBAC: Authorizing `identity:delete_user(user_id=248768777c88479a948983e81b5ccad4)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:14:02.538867 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e138f4d9-4a1e-4ab1-ae40-72ecb6cdd124 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:14:02.573549 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:14:02.587139 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e138f4d9-4a1e-4ab1-ae40-72ecb6cdd124 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '248768777c88479a948983e81b5ccad4'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:02.594510 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e138f4d9-4a1e-4ab1-ae40-72ecb6cdd124 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '248768777c88479a948983e81b5ccad4'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:02.607983 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e138f4d9-4a1e-4ab1-ae40-72ecb6cdd124 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '248768777c88479a948983e81b5ccad4'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:02.609585 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e138f4d9-4a1e-4ab1-ae40-72ecb6cdd124 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '248768777c88479a948983e81b5ccad4'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:02.609585 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e138f4d9-4a1e-4ab1-ae40-72ecb6cdd124 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '248768777c88479a948983e81b5ccad4'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:02.621796 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1296/2622] 10.222.0.99 () {58 vars in 1223 bytes} [Thu Jul 14 16:14:02 2022] DELETE /identity/v3/users/248768777c88479a948983e81b5ccad4 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:14:02.900893 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75088371-eede-49df-a2cc-6eb7d001cad4 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:14:02.902404 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-75088371-eede-49df-a2cc-6eb7d001cad4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:14:02.915309 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75088371-eede-49df-a2cc-6eb7d001cad4 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:14:02.974838 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd11095-a801-498b-9040-ade51a561050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:14:02.975844 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd11095-a801-498b-9040-ade51a561050 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:02.976137 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd11095-a801-498b-9040-ade51a561050 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:02.976416 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd11095-a801-498b-9040-ade51a561050 admin admin] PATH_INFO: `/v3/projects/4504d7ba218f43bf93d535ad1b9aa8f0` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:02.977635 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd11095-a801-498b-9040-ade51a561050 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4504d7ba218f43bf93d535ad1b9aa8f0)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:14:02.979962 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd11095-a801-498b-9040-ade51a561050 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:14:03.027284 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-8bd11095-a801-498b-9040-ade51a561050 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4504d7ba218f43bf93d535ad1b9aa8f0', 'request_id': 'req-8bd11095-a801-498b-9040-ade51a561050'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:03.028420 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:14:03.035972 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-8bd11095-a801-498b-9040-ade51a561050 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4504d7ba218f43bf93d535ad1b9aa8f0', 'request_id': 'req-8bd11095-a801-498b-9040-ade51a561050'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:03.042350 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-8bd11095-a801-498b-9040-ade51a561050 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4504d7ba218f43bf93d535ad1b9aa8f0', 'request_id': 'req-8bd11095-a801-498b-9040-ade51a561050'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:03.056964 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-8bd11095-a801-498b-9040-ade51a561050 admin admin] The token cache is being invalidate because project 4504d7ba218f43bf93d535ad1b9aa8f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 16:14:03.057258 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-8bd11095-a801-498b-9040-ade51a561050 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:03.058406 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-8bd11095-a801-498b-9040-ade51a561050 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:03.060461 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1327/2623] 10.222.0.99 () {58 vars in 1232 bytes} [Thu Jul 14 16:14:02 2022] DELETE /identity/v3/projects/4504d7ba218f43bf93d535ad1b9aa8f0 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:14:03.070750 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f4894b-8be2-48d5-b5cc-9151a0f6b7c0 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:03.071078 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f4894b-8be2-48d5-b5cc-9151a0f6b7c0 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:03.071550 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f4894b-8be2-48d5-b5cc-9151a0f6b7c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:03.092611 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-43f4894b-8be2-48d5-b5cc-9151a0f6b7c0 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:14:03.093845 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-43f4894b-8be2-48d5-b5cc-9151a0f6b7c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:14:03.153897 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1297/2624] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 16:14:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 16:14:03.162739 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4851624a-7f6f-476a-a32e-193d7a1b7db7 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:03.163039 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4851624a-7f6f-476a-a32e-193d7a1b7db7 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:03.163324 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4851624a-7f6f-476a-a32e-193d7a1b7db7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:03.182886 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-4851624a-7f6f-476a-a32e-193d7a1b7db7 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:14:03.184131 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-4851624a-7f6f-476a-a32e-193d7a1b7db7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:14:03.192307 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1328/2625] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 16:14:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:14:03.196633 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f4894b-8be2-48d5-b5cc-9151a0f6b7c0 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:14:03.206387 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f4894b-8be2-48d5-b5cc-9151a0f6b7c0 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:14:03.211248 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d161cb0b-10df-48de-beb7-b8dad27397e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:14:03.211941 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d161cb0b-10df-48de-beb7-b8dad27397e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:03.212232 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d161cb0b-10df-48de-beb7-b8dad27397e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:03.212513 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d161cb0b-10df-48de-beb7-b8dad27397e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:03.213269 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d161cb0b-10df-48de-beb7-b8dad27397e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 16:14:03.213582 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d161cb0b-10df-48de-beb7-b8dad27397e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:14:03.215177 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d161cb0b-10df-48de-beb7-b8dad27397e1 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:14:03.227555 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1298/2626] 10.222.0.99 () {58 vars in 1152 bytes} [Thu Jul 14 16:14:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 16:14:03.231864 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4851624a-7f6f-476a-a32e-193d7a1b7db7 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:14:03.241718 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4851624a-7f6f-476a-a32e-193d7a1b7db7 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:14:03.246423 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b67056a-0079-4e6c-9640-a547c501776d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:14:03.247061 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b67056a-0079-4e6c-9640-a547c501776d admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:03.247347 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b67056a-0079-4e6c-9640-a547c501776d admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:03.247661 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b67056a-0079-4e6c-9640-a547c501776d admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:03.248356 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b67056a-0079-4e6c-9640-a547c501776d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:14:03.250299 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b67056a-0079-4e6c-9640-a547c501776d admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:14:03.275788 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1329/2627] 10.222.0.99 () {60 vars in 1152 bytes} [Thu Jul 14 16:14:03 2022] POST /identity/v3/projects => generated 383 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 16:14:03.279935 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d161cb0b-10df-48de-beb7-b8dad27397e1 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:14:03.291023 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d161cb0b-10df-48de-beb7-b8dad27397e1 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:14:03.348664 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93184d18-e03f-4566-b1d5-b53383279874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:14:03.349347 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93184d18-e03f-4566-b1d5-b53383279874 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:03.349669 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93184d18-e03f-4566-b1d5-b53383279874 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:03.349959 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93184d18-e03f-4566-b1d5-b53383279874 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:03.350678 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93184d18-e03f-4566-b1d5-b53383279874 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:14:03.352648 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93184d18-e03f-4566-b1d5-b53383279874 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:14:03.373391 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1299/2628] 10.222.0.99 () {60 vars in 1143 bytes} [Thu Jul 14 16:14:03 2022] POST /identity/v3/users => generated 292 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 16:14:03.377519 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b67056a-0079-4e6c-9640-a547c501776d admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:14:03.387926 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b67056a-0079-4e6c-9640-a547c501776d admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:14:03.392580 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939800ca-3f3e-409c-b9b2-d3ee3f3372d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:14:03.393242 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939800ca-3f3e-409c-b9b2-d3ee3f3372d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:03.393531 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939800ca-3f3e-409c-b9b2-d3ee3f3372d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:03.393881 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939800ca-3f3e-409c-b9b2-d3ee3f3372d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:03.394665 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939800ca-3f3e-409c-b9b2-d3ee3f3372d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:14:03.396316 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939800ca-3f3e-409c-b9b2-d3ee3f3372d5 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:14:03.407765 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1330/2629] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 16:14:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:14:03.412278 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93184d18-e03f-4566-b1d5-b53383279874 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:14:03.422358 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93184d18-e03f-4566-b1d5-b53383279874 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:14:03.428131 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05267d90-73c6-4a4d-9352-a930a39be1a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:14:03.428785 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05267d90-73c6-4a4d-9352-a930a39be1a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:03.429072 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05267d90-73c6-4a4d-9352-a930a39be1a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:03.429351 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05267d90-73c6-4a4d-9352-a930a39be1a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:03.430077 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05267d90-73c6-4a4d-9352-a930a39be1a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:14:03.431699 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05267d90-73c6-4a4d-9352-a930a39be1a9 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:14:03.441766 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1300/2630] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 16:14:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:14:03.445976 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939800ca-3f3e-409c-b9b2-d3ee3f3372d5 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:14:03.455878 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939800ca-3f3e-409c-b9b2-d3ee3f3372d5 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:14:03.461506 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b354017-ce16-4284-ac21-d4f9667f0de6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:14:03.462390 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b354017-ce16-4284-ac21-d4f9667f0de6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:03.462681 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b354017-ce16-4284-ac21-d4f9667f0de6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:03.462963 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b354017-ce16-4284-ac21-d4f9667f0de6 admin admin] PATH_INFO: `/v3/projects/d15b3928bb35421fb5f9016fd348609f/users/9d78f83040164e2db0aafa65a9a6c3f9/roles/b8c98fddc0fd4436b577fd1d3361bc38` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:03.478554 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b354017-ce16-4284-ac21-d4f9667f0de6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d15b3928bb35421fb5f9016fd348609f, user_id=9d78f83040164e2db0aafa65a9a6c3f9, role_id=b8c98fddc0fd4436b577fd1d3361bc38)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:14:03.481249 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b354017-ce16-4284-ac21-d4f9667f0de6 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:14:03.494410 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1331/2631] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 16:14:03 2022] PUT /identity/v3/projects/d15b3928bb35421fb5f9016fd348609f/users/9d78f83040164e2db0aafa65a9a6c3f9/roles/b8c98fddc0fd4436b577fd1d3361bc38 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:14:03.506473 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608beb1f-7144-4dc7-afb0-c99a6913cf22 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:03.506989 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608beb1f-7144-4dc7-afb0-c99a6913cf22 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:03.507096 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608beb1f-7144-4dc7-afb0-c99a6913cf22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:03.533678 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-608beb1f-7144-4dc7-afb0-c99a6913cf22 None None] MFA Rules not processed for user `9d78f83040164e2db0aafa65a9a6c3f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:14:03.535627 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-608beb1f-7144-4dc7-afb0-c99a6913cf22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:14:03.539638 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:14:03.605173 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1301/2632] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 16:14:03 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:14:03.609832 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b354017-ce16-4284-ac21-d4f9667f0de6 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:14:03.671506 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b354017-ce16-4284-ac21-d4f9667f0de6 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:14:03.676345 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a063be5b-4b4f-4bcd-933f-b1be4a8d3d45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:14:03.677001 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a063be5b-4b4f-4bcd-933f-b1be4a8d3d45 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:03.677287 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a063be5b-4b4f-4bcd-933f-b1be4a8d3d45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:03.677563 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a063be5b-4b4f-4bcd-933f-b1be4a8d3d45 admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:03.678247 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a063be5b-4b4f-4bcd-933f-b1be4a8d3d45 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:14:03.680185 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a063be5b-4b4f-4bcd-933f-b1be4a8d3d45 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:14:03.705662 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1332/2633] 10.222.0.99 () {60 vars in 1152 bytes} [Thu Jul 14 16:14:03 2022] POST /identity/v3/projects => generated 385 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 16:14:03.710485 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608beb1f-7144-4dc7-afb0-c99a6913cf22 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:14:03.721106 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608beb1f-7144-4dc7-afb0-c99a6913cf22 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:14:03.778129 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5de531a-7bea-470b-998e-34087ddafc56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:14:03.778838 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5de531a-7bea-470b-998e-34087ddafc56 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:03.779129 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5de531a-7bea-470b-998e-34087ddafc56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:03.779409 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5de531a-7bea-470b-998e-34087ddafc56 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:03.780137 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5de531a-7bea-470b-998e-34087ddafc56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:14:03.782068 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5de531a-7bea-470b-998e-34087ddafc56 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:14:03.804679 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1302/2634] 10.222.0.99 () {60 vars in 1143 bytes} [Thu Jul 14 16:14:03 2022] POST /identity/v3/users => generated 299 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 16:14:03.808419 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a063be5b-4b4f-4bcd-933f-b1be4a8d3d45 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:14:03.819100 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a063be5b-4b4f-4bcd-933f-b1be4a8d3d45 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:14:03.823697 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca150967-9fa0-468a-aa4f-5ca89a2d78a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:14:03.824351 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca150967-9fa0-468a-aa4f-5ca89a2d78a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:03.824635 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca150967-9fa0-468a-aa4f-5ca89a2d78a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:03.824911 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca150967-9fa0-468a-aa4f-5ca89a2d78a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:03.825632 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca150967-9fa0-468a-aa4f-5ca89a2d78a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:14:03.827215 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca150967-9fa0-468a-aa4f-5ca89a2d78a8 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:14:03.838199 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1333/2635] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 16:14:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:14:03.842306 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5de531a-7bea-470b-998e-34087ddafc56 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:14:03.852431 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5de531a-7bea-470b-998e-34087ddafc56 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:14:03.857008 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2317fec7-e8a4-42f1-87c5-2978170b5d59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:14:03.857878 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2317fec7-e8a4-42f1-87c5-2978170b5d59 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:03.858159 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2317fec7-e8a4-42f1-87c5-2978170b5d59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:03.858433 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2317fec7-e8a4-42f1-87c5-2978170b5d59 admin admin] PATH_INFO: `/v3/projects/b3d739cdf69e4f5c9dff3c5d6881d0e3/users/cad47beff9c24159b38d278e23e0a3d5/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:03.876029 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2317fec7-e8a4-42f1-87c5-2978170b5d59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3d739cdf69e4f5c9dff3c5d6881d0e3, user_id=cad47beff9c24159b38d278e23e0a3d5, role_id=d0f6ba1314f14d9f9c816bdb5d0c3aac)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:14:03.880014 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2317fec7-e8a4-42f1-87c5-2978170b5d59 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:14:03.892990 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1303/2636] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 16:14:03 2022] PUT /identity/v3/projects/b3d739cdf69e4f5c9dff3c5d6881d0e3/users/cad47beff9c24159b38d278e23e0a3d5/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:14:03.898226 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca150967-9fa0-468a-aa4f-5ca89a2d78a8 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:14:03.908350 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca150967-9fa0-468a-aa4f-5ca89a2d78a8 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:14:03.953890 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:14:03.969391 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bbf3bc-5d3a-4928-9ca5-7b2ce47453aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:14:03.970117 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bbf3bc-5d3a-4928-9ca5-7b2ce47453aa admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:03.970479 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bbf3bc-5d3a-4928-9ca5-7b2ce47453aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:03.970763 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bbf3bc-5d3a-4928-9ca5-7b2ce47453aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:03.971498 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bbf3bc-5d3a-4928-9ca5-7b2ce47453aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:14:03.973669 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bbf3bc-5d3a-4928-9ca5-7b2ce47453aa admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:14:03.984092 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1334/2637] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 16:14:03 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:14:03.988744 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2317fec7-e8a4-42f1-87c5-2978170b5d59 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:14:03.999399 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2317fec7-e8a4-42f1-87c5-2978170b5d59 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:14:04.004908 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d57fc0e-194e-4a3b-bbaa-6a2432099054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:14:04.005801 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d57fc0e-194e-4a3b-bbaa-6a2432099054 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:04.006120 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d57fc0e-194e-4a3b-bbaa-6a2432099054 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:04.006402 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d57fc0e-194e-4a3b-bbaa-6a2432099054 admin admin] PATH_INFO: `/v3/projects/b3d739cdf69e4f5c9dff3c5d6881d0e3/users/cad47beff9c24159b38d278e23e0a3d5/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:04.008568 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d57fc0e-194e-4a3b-bbaa-6a2432099054 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3d739cdf69e4f5c9dff3c5d6881d0e3, user_id=cad47beff9c24159b38d278e23e0a3d5, role_id=d0f6ba1314f14d9f9c816bdb5d0c3aac)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:14:04.011209 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d57fc0e-194e-4a3b-bbaa-6a2432099054 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:14:04.022321 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1304/2638] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 16:14:03 2022] PUT /identity/v3/projects/b3d739cdf69e4f5c9dff3c5d6881d0e3/users/cad47beff9c24159b38d278e23e0a3d5/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:14:04.032800 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c999c990-a364-4da7-98db-b20c9ce8f475 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:04.033102 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c999c990-a364-4da7-98db-b20c9ce8f475 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:04.033390 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c999c990-a364-4da7-98db-b20c9ce8f475 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:04.060238 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-c999c990-a364-4da7-98db-b20c9ce8f475 None None] MFA Rules not processed for user `cad47beff9c24159b38d278e23e0a3d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:14:04.061463 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-c999c990-a364-4da7-98db-b20c9ce8f475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:14:04.130057 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1335/2639] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 16:14:04 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:14:04.140959 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d57fc0e-194e-4a3b-bbaa-6a2432099054 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:14:04.142658 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0d57fc0e-194e-4a3b-bbaa-6a2432099054 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:14:04.199857 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d57fc0e-194e-4a3b-bbaa-6a2432099054 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:14:04.203428 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ca4dda-b98e-429d-befa-70868e59c705 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3e1a2b83f3d4d88ba5084dfaffd22f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:14:04.204173 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ca4dda-b98e-429d-befa-70868e59c705 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:04.204392 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ca4dda-b98e-429d-befa-70868e59c705 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:04.204670 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ca4dda-b98e-429d-befa-70868e59c705 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:04.215092 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:14:04.262903 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ca4dda-b98e-429d-befa-70868e59c705 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:14:04.264376 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ca4dda-b98e-429d-befa-70868e59c705 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:14:04.274184 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1305/2640] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:14:04 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:14:08.500126 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c999c990-a364-4da7-98db-b20c9ce8f475 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:14:08.501532 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-c999c990-a364-4da7-98db-b20c9ce8f475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:14:08.525810 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c999c990-a364-4da7-98db-b20c9ce8f475 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:14:08.529348 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4456edc-cb70-4de1-845d-b8e0be579170 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3e1a2b83f3d4d88ba5084dfaffd22f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:14:08.529996 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4456edc-cb70-4de1-845d-b8e0be579170 service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:08.530281 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4456edc-cb70-4de1-845d-b8e0be579170 service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:08.530560 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4456edc-cb70-4de1-845d-b8e0be579170 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:08.540325 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4456edc-cb70-4de1-845d-b8e0be579170 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:14:08.541749 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4456edc-cb70-4de1-845d-b8e0be579170 service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:14:08.552853 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1336/2641] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:14:08 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:14:13.445128 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ca4dda-b98e-429d-befa-70868e59c705 service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:14:13.508714 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ca4dda-b98e-429d-befa-70868e59c705 service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:14:13.513164 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c567ea-10ae-4b83-a0f1-f5f16458bd9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:14:13.514032 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c567ea-10ae-4b83-a0f1-f5f16458bd9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:13.514316 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c567ea-10ae-4b83-a0f1-f5f16458bd9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:13.514596 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c567ea-10ae-4b83-a0f1-f5f16458bd9a admin admin] PATH_INFO: `/v3/users/9d78f83040164e2db0aafa65a9a6c3f9` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:13.515809 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c567ea-10ae-4b83-a0f1-f5f16458bd9a admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d78f83040164e2db0aafa65a9a6c3f9)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:14:13.517743 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c567ea-10ae-4b83-a0f1-f5f16458bd9a admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:14:13.569137 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-60c567ea-10ae-4b83-a0f1-f5f16458bd9a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d78f83040164e2db0aafa65a9a6c3f9'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:13.576219 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-60c567ea-10ae-4b83-a0f1-f5f16458bd9a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d78f83040164e2db0aafa65a9a6c3f9'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:13.588443 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-60c567ea-10ae-4b83-a0f1-f5f16458bd9a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d78f83040164e2db0aafa65a9a6c3f9'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:13.589102 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-60c567ea-10ae-4b83-a0f1-f5f16458bd9a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d78f83040164e2db0aafa65a9a6c3f9'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:13.589721 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-60c567ea-10ae-4b83-a0f1-f5f16458bd9a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d78f83040164e2db0aafa65a9a6c3f9'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:13.602575 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1306/2642] 10.222.0.99 () {58 vars in 1223 bytes} [Thu Jul 14 16:14:13 2022] DELETE /identity/v3/users/9d78f83040164e2db0aafa65a9a6c3f9 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:14:13.607150 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68233836-e988-4559-95b6-784d7d0edfc4 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:13.607636 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68233836-e988-4559-95b6-784d7d0edfc4 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:13.607763 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68233836-e988-4559-95b6-784d7d0edfc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:13.626015 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-68233836-e988-4559-95b6-784d7d0edfc4 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:14:13.627217 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-68233836-e988-4559-95b6-784d7d0edfc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:14:13.690712 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1337/2643] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 16:14:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:14:13.700264 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c567ea-10ae-4b83-a0f1-f5f16458bd9a admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:14:13.701660 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-60c567ea-10ae-4b83-a0f1-f5f16458bd9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:14:13.737114 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:14:13.764640 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c567ea-10ae-4b83-a0f1-f5f16458bd9a admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:14:13.768750 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26db49de-6b06-4277-849c-5d3dc5d5f902 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:14:13.769542 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26db49de-6b06-4277-849c-5d3dc5d5f902 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:13.769974 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26db49de-6b06-4277-849c-5d3dc5d5f902 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:13.770257 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26db49de-6b06-4277-849c-5d3dc5d5f902 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:13.780036 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26db49de-6b06-4277-849c-5d3dc5d5f902 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:14:13.781476 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26db49de-6b06-4277-849c-5d3dc5d5f902 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:14:13.791912 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1307/2644] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:14:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:14:14.066858 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68233836-e988-4559-95b6-784d7d0edfc4 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:14:14.068374 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-68233836-e988-4559-95b6-784d7d0edfc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:14:14.080767 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68233836-e988-4559-95b6-784d7d0edfc4 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:14:14.085542 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9f4691-3f8d-420b-b708-5cbe6b231fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:14:14.086410 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9f4691-3f8d-420b-b708-5cbe6b231fde admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:14.086739 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9f4691-3f8d-420b-b708-5cbe6b231fde admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:14.087019 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9f4691-3f8d-420b-b708-5cbe6b231fde admin admin] PATH_INFO: `/v3/projects/d15b3928bb35421fb5f9016fd348609f` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:14.088220 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9f4691-3f8d-420b-b708-5cbe6b231fde admin admin] RBAC: Authorizing `identity:delete_project(project_id=d15b3928bb35421fb5f9016fd348609f)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:14:14.090144 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9f4691-3f8d-420b-b708-5cbe6b231fde admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:14:14.134269 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:14:14.141222 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-1c9f4691-3f8d-420b-b708-5cbe6b231fde admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd15b3928bb35421fb5f9016fd348609f', 'request_id': 'req-1c9f4691-3f8d-420b-b708-5cbe6b231fde'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:14.147866 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-1c9f4691-3f8d-420b-b708-5cbe6b231fde admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd15b3928bb35421fb5f9016fd348609f', 'request_id': 'req-1c9f4691-3f8d-420b-b708-5cbe6b231fde'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:14.153392 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-1c9f4691-3f8d-420b-b708-5cbe6b231fde admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd15b3928bb35421fb5f9016fd348609f', 'request_id': 'req-1c9f4691-3f8d-420b-b708-5cbe6b231fde'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:14.166094 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-1c9f4691-3f8d-420b-b708-5cbe6b231fde admin admin] The token cache is being invalidate because project d15b3928bb35421fb5f9016fd348609f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 16:14:14.166406 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-1c9f4691-3f8d-420b-b708-5cbe6b231fde admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:14.167051 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-1c9f4691-3f8d-420b-b708-5cbe6b231fde admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:14.168627 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1338/2645] 10.222.0.99 () {58 vars in 1232 bytes} [Thu Jul 14 16:14:14 2022] DELETE /identity/v3/projects/d15b3928bb35421fb5f9016fd348609f => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:14:14.173061 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26db49de-6b06-4277-849c-5d3dc5d5f902 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:14:14.183904 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26db49de-6b06-4277-849c-5d3dc5d5f902 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:14:14.240632 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c790aa63-9092-493b-827d-0c434f2c27b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:14:14.241433 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c790aa63-9092-493b-827d-0c434f2c27b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:14.241722 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c790aa63-9092-493b-827d-0c434f2c27b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:14.242001 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c790aa63-9092-493b-827d-0c434f2c27b2 admin admin] PATH_INFO: `/v3/users/cad47beff9c24159b38d278e23e0a3d5` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:14.243189 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c790aa63-9092-493b-827d-0c434f2c27b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cad47beff9c24159b38d278e23e0a3d5)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:14:14.245149 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c790aa63-9092-493b-827d-0c434f2c27b2 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:14:14.291811 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-c790aa63-9092-493b-827d-0c434f2c27b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cad47beff9c24159b38d278e23e0a3d5'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:14.300453 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-c790aa63-9092-493b-827d-0c434f2c27b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cad47beff9c24159b38d278e23e0a3d5'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:14.312584 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-c790aa63-9092-493b-827d-0c434f2c27b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cad47beff9c24159b38d278e23e0a3d5'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:14.313224 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-c790aa63-9092-493b-827d-0c434f2c27b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cad47beff9c24159b38d278e23e0a3d5'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:14.313827 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-c790aa63-9092-493b-827d-0c434f2c27b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cad47beff9c24159b38d278e23e0a3d5'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:14.326524 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1308/2646] 10.222.0.99 () {58 vars in 1223 bytes} [Thu Jul 14 16:14:14 2022] DELETE /identity/v3/users/cad47beff9c24159b38d278e23e0a3d5 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:14:14.597242 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9f4691-3f8d-420b-b708-5cbe6b231fde admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:14:14.599240 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-1c9f4691-3f8d-420b-b708-5cbe6b231fde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:14:14.614050 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9f4691-3f8d-420b-b708-5cbe6b231fde admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:14:14.684057 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c863b53c-7910-4ded-8543-06d13283bce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:14:14.685829 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c863b53c-7910-4ded-8543-06d13283bce8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:14.686154 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c863b53c-7910-4ded-8543-06d13283bce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:14.686476 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c863b53c-7910-4ded-8543-06d13283bce8 admin admin] PATH_INFO: `/v3/projects/b3d739cdf69e4f5c9dff3c5d6881d0e3` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:14.688681 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c863b53c-7910-4ded-8543-06d13283bce8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3d739cdf69e4f5c9dff3c5d6881d0e3)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:14:14.689808 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c863b53c-7910-4ded-8543-06d13283bce8 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:14:14.747095 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-c863b53c-7910-4ded-8543-06d13283bce8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3d739cdf69e4f5c9dff3c5d6881d0e3', 'request_id': 'req-c863b53c-7910-4ded-8543-06d13283bce8'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:14.754855 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-c863b53c-7910-4ded-8543-06d13283bce8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3d739cdf69e4f5c9dff3c5d6881d0e3', 'request_id': 'req-c863b53c-7910-4ded-8543-06d13283bce8'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:14.760891 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-c863b53c-7910-4ded-8543-06d13283bce8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3d739cdf69e4f5c9dff3c5d6881d0e3', 'request_id': 'req-c863b53c-7910-4ded-8543-06d13283bce8'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:14.774209 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-c863b53c-7910-4ded-8543-06d13283bce8 admin admin] The token cache is being invalidate because project b3d739cdf69e4f5c9dff3c5d6881d0e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 16:14:14.774386 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-c863b53c-7910-4ded-8543-06d13283bce8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:14.774918 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-c863b53c-7910-4ded-8543-06d13283bce8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:14.776508 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1339/2647] 10.222.0.99 () {58 vars in 1232 bytes} [Thu Jul 14 16:14:14 2022] DELETE /identity/v3/projects/b3d739cdf69e4f5c9dff3c5d6881d0e3 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:14:14.782437 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5477d4a9-1e3a-4cc7-98f2-8242f0cd5344 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:14.782915 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5477d4a9-1e3a-4cc7-98f2-8242f0cd5344 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:14.783021 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5477d4a9-1e3a-4cc7-98f2-8242f0cd5344 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:14.802165 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-5477d4a9-1e3a-4cc7-98f2-8242f0cd5344 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:14:14.803551 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-5477d4a9-1e3a-4cc7-98f2-8242f0cd5344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:14:14.853870 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:14:14.868498 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1309/2648] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 16:14:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:14:14.877799 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e8727f-0bfb-455a-a315-1c652c6afc8e None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:14.878690 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e8727f-0bfb-455a-a315-1c652c6afc8e None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:14.878975 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e8727f-0bfb-455a-a315-1c652c6afc8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:14.898363 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-85e8727f-0bfb-455a-a315-1c652c6afc8e None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:14:14.899557 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-85e8727f-0bfb-455a-a315-1c652c6afc8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:14:14.907359 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1340/2649] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 16:14:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:14:14.911885 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5477d4a9-1e3a-4cc7-98f2-8242f0cd5344 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:14:14.922033 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5477d4a9-1e3a-4cc7-98f2-8242f0cd5344 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:14:14.927256 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5afc63f-0997-486f-91f6-ee7e89c1ddcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:14:14.928003 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5afc63f-0997-486f-91f6-ee7e89c1ddcf admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:14.928294 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5afc63f-0997-486f-91f6-ee7e89c1ddcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:14.928575 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5afc63f-0997-486f-91f6-ee7e89c1ddcf admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:14.929310 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5afc63f-0997-486f-91f6-ee7e89c1ddcf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 16:14:14.929624 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5afc63f-0997-486f-91f6-ee7e89c1ddcf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:14:14.931225 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5afc63f-0997-486f-91f6-ee7e89c1ddcf admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:14:14.943426 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1310/2650] 10.222.0.99 () {58 vars in 1152 bytes} [Thu Jul 14 16:14:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 16:14:14.947852 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e8727f-0bfb-455a-a315-1c652c6afc8e None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:14:14.957109 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e8727f-0bfb-455a-a315-1c652c6afc8e None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:14:14.961772 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126a93b7-689f-4f52-b730-ac73f4146803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:14:14.962427 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126a93b7-689f-4f52-b730-ac73f4146803 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:14.962714 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126a93b7-689f-4f52-b730-ac73f4146803 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:14.963025 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126a93b7-689f-4f52-b730-ac73f4146803 admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:14.963767 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126a93b7-689f-4f52-b730-ac73f4146803 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:14:14.965687 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126a93b7-689f-4f52-b730-ac73f4146803 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:14:14.998961 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1341/2651] 10.222.0.99 () {60 vars in 1152 bytes} [Thu Jul 14 16:14:14 2022] POST /identity/v3/projects => generated 361 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 16:14:15.000107 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5afc63f-0997-486f-91f6-ee7e89c1ddcf admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:14:15.011053 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5afc63f-0997-486f-91f6-ee7e89c1ddcf admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:14:15.072896 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b89cd37-9834-482b-980b-cc02ba60e442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:14:15.073495 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b89cd37-9834-482b-980b-cc02ba60e442 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:15.073788 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b89cd37-9834-482b-980b-cc02ba60e442 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:15.074073 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b89cd37-9834-482b-980b-cc02ba60e442 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:15.074779 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b89cd37-9834-482b-980b-cc02ba60e442 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:14:15.076771 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b89cd37-9834-482b-980b-cc02ba60e442 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:14:15.098102 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1311/2652] 10.222.0.99 () {60 vars in 1143 bytes} [Thu Jul 14 16:14:14 2022] POST /identity/v3/users => generated 281 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 16:14:15.102244 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126a93b7-689f-4f52-b730-ac73f4146803 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:14:15.113264 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126a93b7-689f-4f52-b730-ac73f4146803 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:14:15.118027 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e629c5b-1458-4efd-81c8-241158c1e193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:14:15.118684 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e629c5b-1458-4efd-81c8-241158c1e193 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:15.118979 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e629c5b-1458-4efd-81c8-241158c1e193 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:15.119260 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e629c5b-1458-4efd-81c8-241158c1e193 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:15.120094 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e629c5b-1458-4efd-81c8-241158c1e193 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:14:15.121690 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e629c5b-1458-4efd-81c8-241158c1e193 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:14:15.132193 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1342/2653] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 16:14:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:14:15.136282 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b89cd37-9834-482b-980b-cc02ba60e442 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:14:15.146945 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b89cd37-9834-482b-980b-cc02ba60e442 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:14:15.151518 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689122d5-30b3-47c4-8876-4803a95d9409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:14:15.152208 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689122d5-30b3-47c4-8876-4803a95d9409 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:15.152496 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689122d5-30b3-47c4-8876-4803a95d9409 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:15.152779 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689122d5-30b3-47c4-8876-4803a95d9409 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:15.153507 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689122d5-30b3-47c4-8876-4803a95d9409 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:14:15.155265 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689122d5-30b3-47c4-8876-4803a95d9409 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:14:15.166608 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1312/2654] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 16:14:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 16:14:15.170816 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e629c5b-1458-4efd-81c8-241158c1e193 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:14:15.181727 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e629c5b-1458-4efd-81c8-241158c1e193 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:14:15.186575 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d0f206-ec28-466f-8fc9-89fabc01c46a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:14:15.187458 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d0f206-ec28-466f-8fc9-89fabc01c46a admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:15.187780 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d0f206-ec28-466f-8fc9-89fabc01c46a admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:15.188062 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d0f206-ec28-466f-8fc9-89fabc01c46a admin admin] PATH_INFO: `/v3/projects/4dc9983f52fe4006979b01955c1fadbd/users/63c93383da1e428bbd29976e41585aea/roles/b8c98fddc0fd4436b577fd1d3361bc38` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:15.202545 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d0f206-ec28-466f-8fc9-89fabc01c46a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4dc9983f52fe4006979b01955c1fadbd, user_id=63c93383da1e428bbd29976e41585aea, role_id=b8c98fddc0fd4436b577fd1d3361bc38)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:14:15.205237 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d0f206-ec28-466f-8fc9-89fabc01c46a admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:14:15.208496 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:14:15.220619 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1343/2655] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 16:14:15 2022] PUT /identity/v3/projects/4dc9983f52fe4006979b01955c1fadbd/users/63c93383da1e428bbd29976e41585aea/roles/b8c98fddc0fd4436b577fd1d3361bc38 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:14:15.236253 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e3b40d-681f-4f7f-b8c3-f3940a3f5441 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:15.236551 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e3b40d-681f-4f7f-b8c3-f3940a3f5441 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:15.236837 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e3b40d-681f-4f7f-b8c3-f3940a3f5441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:15.266462 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-f6e3b40d-681f-4f7f-b8c3-f3940a3f5441 None None] MFA Rules not processed for user `63c93383da1e428bbd29976e41585aea`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:14:15.267730 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f6e3b40d-681f-4f7f-b8c3-f3940a3f5441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:14:15.333371 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1313/2656] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 16:14:15 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:14:15.364486 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d0f206-ec28-466f-8fc9-89fabc01c46a admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:14:15.366187 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-92d0f206-ec28-466f-8fc9-89fabc01c46a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:14:15.423618 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d0f206-ec28-466f-8fc9-89fabc01c46a admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:14:15.428226 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b6d3eb-ebb1-4d68-ad06-e308e47b422e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3e1a2b83f3d4d88ba5084dfaffd22f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:14:15.428881 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b6d3eb-ebb1-4d68-ad06-e308e47b422e service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:15.429172 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b6d3eb-ebb1-4d68-ad06-e308e47b422e service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:15.429548 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b6d3eb-ebb1-4d68-ad06-e308e47b422e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:15.438084 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b6d3eb-ebb1-4d68-ad06-e308e47b422e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:14:15.439519 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b6d3eb-ebb1-4d68-ad06-e308e47b422e service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:14:15.449086 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1344/2657] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:14:15 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:14:15.463312 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e3b40d-681f-4f7f-b8c3-f3940a3f5441 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:14:15.511208 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:14:15.525899 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e3b40d-681f-4f7f-b8c3-f3940a3f5441 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:14:15.530397 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66b6c01-9455-4d27-9b6b-9567e90dcc18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:14:15.531263 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66b6c01-9455-4d27-9b6b-9567e90dcc18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:15.531548 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66b6c01-9455-4d27-9b6b-9567e90dcc18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:15.531858 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66b6c01-9455-4d27-9b6b-9567e90dcc18 admin admin] PATH_INFO: `/v3/users/63c93383da1e428bbd29976e41585aea` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:15.533047 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66b6c01-9455-4d27-9b6b-9567e90dcc18 admin admin] RBAC: Authorizing `identity:delete_user(user_id=63c93383da1e428bbd29976e41585aea)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:14:15.535258 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66b6c01-9455-4d27-9b6b-9567e90dcc18 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:14:15.581970 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e66b6c01-9455-4d27-9b6b-9567e90dcc18 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63c93383da1e428bbd29976e41585aea'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:15.589434 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e66b6c01-9455-4d27-9b6b-9567e90dcc18 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63c93383da1e428bbd29976e41585aea'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:15.601597 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e66b6c01-9455-4d27-9b6b-9567e90dcc18 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63c93383da1e428bbd29976e41585aea'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:15.602262 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e66b6c01-9455-4d27-9b6b-9567e90dcc18 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63c93383da1e428bbd29976e41585aea'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:15.602892 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e66b6c01-9455-4d27-9b6b-9567e90dcc18 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63c93383da1e428bbd29976e41585aea'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:15.615688 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1314/2658] 10.222.0.99 () {58 vars in 1223 bytes} [Thu Jul 14 16:14:15 2022] DELETE /identity/v3/users/63c93383da1e428bbd29976e41585aea => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:14:15.620223 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554e61c1-d484-4744-862b-2e75f41ae238 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:15.620742 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554e61c1-d484-4744-862b-2e75f41ae238 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:15.620882 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554e61c1-d484-4744-862b-2e75f41ae238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:15.638322 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-554e61c1-d484-4744-862b-2e75f41ae238 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:14:15.639533 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-554e61c1-d484-4744-862b-2e75f41ae238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:14:15.702535 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1345/2659] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 16:14:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:14:15.712242 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66b6c01-9455-4d27-9b6b-9567e90dcc18 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:14:15.713661 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e66b6c01-9455-4d27-9b6b-9567e90dcc18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:14:15.785993 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66b6c01-9455-4d27-9b6b-9567e90dcc18 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:14:15.789845 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d42e64-836c-4cc4-90e7-cc571bf6ebf4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:14:15.790721 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d42e64-836c-4cc4-90e7-cc571bf6ebf4 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:15.791158 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d42e64-836c-4cc4-90e7-cc571bf6ebf4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:15.791539 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d42e64-836c-4cc4-90e7-cc571bf6ebf4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:15.808738 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d42e64-836c-4cc4-90e7-cc571bf6ebf4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:14:15.810295 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d42e64-836c-4cc4-90e7-cc571bf6ebf4 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:14:15.821028 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1315/2660] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:14:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:14:16.093593 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554e61c1-d484-4744-862b-2e75f41ae238 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:14:16.095350 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-554e61c1-d484-4744-862b-2e75f41ae238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:14:16.114815 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554e61c1-d484-4744-862b-2e75f41ae238 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:14:16.122285 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2fa686-d6ae-4df7-8bf8-1b0825871945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:14:16.123965 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2fa686-d6ae-4df7-8bf8-1b0825871945 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:16.124560 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2fa686-d6ae-4df7-8bf8-1b0825871945 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:16.125221 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2fa686-d6ae-4df7-8bf8-1b0825871945 admin admin] PATH_INFO: `/v3/projects/4dc9983f52fe4006979b01955c1fadbd` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:16.127785 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2fa686-d6ae-4df7-8bf8-1b0825871945 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4dc9983f52fe4006979b01955c1fadbd)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:14:16.132813 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2fa686-d6ae-4df7-8bf8-1b0825871945 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:14:16.199384 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-3f2fa686-d6ae-4df7-8bf8-1b0825871945 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4dc9983f52fe4006979b01955c1fadbd', 'request_id': 'req-3f2fa686-d6ae-4df7-8bf8-1b0825871945'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:16.206907 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-3f2fa686-d6ae-4df7-8bf8-1b0825871945 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4dc9983f52fe4006979b01955c1fadbd', 'request_id': 'req-3f2fa686-d6ae-4df7-8bf8-1b0825871945'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:16.212939 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-3f2fa686-d6ae-4df7-8bf8-1b0825871945 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4dc9983f52fe4006979b01955c1fadbd', 'request_id': 'req-3f2fa686-d6ae-4df7-8bf8-1b0825871945'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:16.231356 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-3f2fa686-d6ae-4df7-8bf8-1b0825871945 admin admin] The token cache is being invalidate because project 4dc9983f52fe4006979b01955c1fadbd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 16:14:16.231678 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-3f2fa686-d6ae-4df7-8bf8-1b0825871945 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:16.232328 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-3f2fa686-d6ae-4df7-8bf8-1b0825871945 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:16.233937 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1346/2661] 10.222.0.99 () {58 vars in 1232 bytes} [Thu Jul 14 16:14:16 2022] DELETE /identity/v3/projects/4dc9983f52fe4006979b01955c1fadbd => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:14:16.239475 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea25253-e17f-4ce0-a877-b31fd765f1ca None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:16.239795 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea25253-e17f-4ce0-a877-b31fd765f1ca None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:16.240074 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea25253-e17f-4ce0-a877-b31fd765f1ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:16.258548 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-5ea25253-e17f-4ce0-a877-b31fd765f1ca None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:14:16.259867 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-5ea25253-e17f-4ce0-a877-b31fd765f1ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:14:16.320358 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1316/2662] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 16:14:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 16:14:16.328484 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897d987c-016f-4386-9d9a-52ae292eb481 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:16.328779 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897d987c-016f-4386-9d9a-52ae292eb481 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:16.329066 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897d987c-016f-4386-9d9a-52ae292eb481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:16.347642 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-897d987c-016f-4386-9d9a-52ae292eb481 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:14:16.348842 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-897d987c-016f-4386-9d9a-52ae292eb481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:14:16.357495 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1347/2663] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 16:14:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:14:16.361904 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea25253-e17f-4ce0-a877-b31fd765f1ca None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:14:16.372364 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea25253-e17f-4ce0-a877-b31fd765f1ca None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:14:16.376895 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bba895-9000-415f-a858-c9e57b0cc110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:14:16.377550 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bba895-9000-415f-a858-c9e57b0cc110 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:16.377860 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bba895-9000-415f-a858-c9e57b0cc110 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:16.378262 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bba895-9000-415f-a858-c9e57b0cc110 admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:16.379165 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bba895-9000-415f-a858-c9e57b0cc110 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 16:14:16.379475 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bba895-9000-415f-a858-c9e57b0cc110 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:14:16.381103 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bba895-9000-415f-a858-c9e57b0cc110 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:14:16.387906 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:14:16.393917 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1317/2664] 10.222.0.99 () {58 vars in 1152 bytes} [Thu Jul 14 16:14:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 16:14:16.398275 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897d987c-016f-4386-9d9a-52ae292eb481 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:14:16.407815 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897d987c-016f-4386-9d9a-52ae292eb481 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:14:16.414150 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdce906c-57fe-4087-b498-cdbff15f7063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:14:16.414813 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdce906c-57fe-4087-b498-cdbff15f7063 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:16.415100 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdce906c-57fe-4087-b498-cdbff15f7063 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:16.415404 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdce906c-57fe-4087-b498-cdbff15f7063 admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:16.416142 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdce906c-57fe-4087-b498-cdbff15f7063 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:14:16.418048 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdce906c-57fe-4087-b498-cdbff15f7063 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:14:16.432402 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:14:16.445889 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1348/2665] 10.222.0.99 () {60 vars in 1152 bytes} [Thu Jul 14 16:14:16 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 16:14:16.450059 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bba895-9000-415f-a858-c9e57b0cc110 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:14:16.460714 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bba895-9000-415f-a858-c9e57b0cc110 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:14:16.517942 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d368c38f-2f50-4623-ba57-4961371019c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:14:16.518694 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d368c38f-2f50-4623-ba57-4961371019c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:16.518694 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d368c38f-2f50-4623-ba57-4961371019c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:16.519244 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d368c38f-2f50-4623-ba57-4961371019c5 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:16.519877 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d368c38f-2f50-4623-ba57-4961371019c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:14:16.521803 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d368c38f-2f50-4623-ba57-4961371019c5 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:14:16.541809 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1318/2666] 10.222.0.99 () {60 vars in 1143 bytes} [Thu Jul 14 16:14:16 2022] POST /identity/v3/users => generated 287 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 16:14:16.546032 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdce906c-57fe-4087-b498-cdbff15f7063 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:14:16.557810 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdce906c-57fe-4087-b498-cdbff15f7063 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:14:16.562510 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f39865-3ef6-49f7-99a7-bcfcfabb1119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:14:16.563172 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f39865-3ef6-49f7-99a7-bcfcfabb1119 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:16.563462 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f39865-3ef6-49f7-99a7-bcfcfabb1119 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:16.563767 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f39865-3ef6-49f7-99a7-bcfcfabb1119 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:16.564534 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f39865-3ef6-49f7-99a7-bcfcfabb1119 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:14:16.566681 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f39865-3ef6-49f7-99a7-bcfcfabb1119 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:14:16.577185 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1349/2667] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 16:14:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:14:16.581762 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d368c38f-2f50-4623-ba57-4961371019c5 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:14:16.592013 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d368c38f-2f50-4623-ba57-4961371019c5 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:14:16.596667 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c0be5e-dd37-4e00-a517-8a66e26ea9ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:14:16.597829 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c0be5e-dd37-4e00-a517-8a66e26ea9ab admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:16.598114 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c0be5e-dd37-4e00-a517-8a66e26ea9ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:16.598397 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c0be5e-dd37-4e00-a517-8a66e26ea9ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:16.599191 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c0be5e-dd37-4e00-a517-8a66e26ea9ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:14:16.600757 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c0be5e-dd37-4e00-a517-8a66e26ea9ab admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:14:16.610956 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1319/2668] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 16:14:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:14:16.615062 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f39865-3ef6-49f7-99a7-bcfcfabb1119 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:14:16.625033 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f39865-3ef6-49f7-99a7-bcfcfabb1119 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:14:16.629850 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eea5f95-0467-42b9-9fbc-cccdda505558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:14:16.630723 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eea5f95-0467-42b9-9fbc-cccdda505558 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:16.631009 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eea5f95-0467-42b9-9fbc-cccdda505558 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:16.631290 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eea5f95-0467-42b9-9fbc-cccdda505558 admin admin] PATH_INFO: `/v3/projects/59d3e42c3b494b34b9eba6a3f682a632/users/9b2e46a6e14c432193583104d30f4768/roles/b8c98fddc0fd4436b577fd1d3361bc38` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:16.646453 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eea5f95-0467-42b9-9fbc-cccdda505558 admin admin] RBAC: Authorizing `identity:create_grant(project_id=59d3e42c3b494b34b9eba6a3f682a632, user_id=9b2e46a6e14c432193583104d30f4768, role_id=b8c98fddc0fd4436b577fd1d3361bc38)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:14:16.651093 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eea5f95-0467-42b9-9fbc-cccdda505558 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:14:16.664719 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1350/2669] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 16:14:16 2022] PUT /identity/v3/projects/59d3e42c3b494b34b9eba6a3f682a632/users/9b2e46a6e14c432193583104d30f4768/roles/b8c98fddc0fd4436b577fd1d3361bc38 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:14:16.676582 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189a9b98-fcc7-4f66-ba44-e77f5cb16bc8 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:16.676879 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189a9b98-fcc7-4f66-ba44-e77f5cb16bc8 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:16.677164 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189a9b98-fcc7-4f66-ba44-e77f5cb16bc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:16.704734 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-189a9b98-fcc7-4f66-ba44-e77f5cb16bc8 None None] MFA Rules not processed for user `9b2e46a6e14c432193583104d30f4768`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:14:16.706632 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-189a9b98-fcc7-4f66-ba44-e77f5cb16bc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:14:16.778872 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1320/2670] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 16:14:16 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 16:14:16.791920 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eea5f95-0467-42b9-9fbc-cccdda505558 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:14:16.794223 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-4eea5f95-0467-42b9-9fbc-cccdda505558 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:14:16.862058 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eea5f95-0467-42b9-9fbc-cccdda505558 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:14:16.866494 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe50a7b4-e5f4-4814-b2e3-2690b952131e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3e1a2b83f3d4d88ba5084dfaffd22f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:14:16.867277 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe50a7b4-e5f4-4814-b2e3-2690b952131e service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:16.867570 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe50a7b4-e5f4-4814-b2e3-2690b952131e service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:16.868013 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe50a7b4-e5f4-4814-b2e3-2690b952131e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:16.877002 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe50a7b4-e5f4-4814-b2e3-2690b952131e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:14:16.879140 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe50a7b4-e5f4-4814-b2e3-2690b952131e service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:14:16.888460 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1351/2671] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:14:16 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 16:14:19.922426 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189a9b98-fcc7-4f66-ba44-e77f5cb16bc8 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:14:19.987173 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189a9b98-fcc7-4f66-ba44-e77f5cb16bc8 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:14:19.992619 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f452d571-bff5-4d99-893e-b91a772cf6bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:14:19.993589 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f452d571-bff5-4d99-893e-b91a772cf6bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:19.993952 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f452d571-bff5-4d99-893e-b91a772cf6bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:19.994303 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f452d571-bff5-4d99-893e-b91a772cf6bb admin admin] PATH_INFO: `/v3/users/9b2e46a6e14c432193583104d30f4768` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:19.995608 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f452d571-bff5-4d99-893e-b91a772cf6bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b2e46a6e14c432193583104d30f4768)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:14:19.997615 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f452d571-bff5-4d99-893e-b91a772cf6bb admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:14:20.004087 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:14:20.045300 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-f452d571-bff5-4d99-893e-b91a772cf6bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b2e46a6e14c432193583104d30f4768'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:20.052787 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-f452d571-bff5-4d99-893e-b91a772cf6bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b2e46a6e14c432193583104d30f4768'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:20.064425 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-f452d571-bff5-4d99-893e-b91a772cf6bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b2e46a6e14c432193583104d30f4768'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:20.065044 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-f452d571-bff5-4d99-893e-b91a772cf6bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b2e46a6e14c432193583104d30f4768'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:20.066239 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-f452d571-bff5-4d99-893e-b91a772cf6bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b2e46a6e14c432193583104d30f4768'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:20.078779 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1321/2672] 10.222.0.99 () {58 vars in 1223 bytes} [Thu Jul 14 16:14:19 2022] DELETE /identity/v3/users/9b2e46a6e14c432193583104d30f4768 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:14:20.084760 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679f09a0-27d1-4334-8b60-56ab4babbbd7 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:20.085272 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679f09a0-27d1-4334-8b60-56ab4babbbd7 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:20.085390 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679f09a0-27d1-4334-8b60-56ab4babbbd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:20.102670 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-679f09a0-27d1-4334-8b60-56ab4babbbd7 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:14:20.103909 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-679f09a0-27d1-4334-8b60-56ab4babbbd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:14:20.163345 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1352/2673] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 16:14:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:14:20.173407 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f452d571-bff5-4d99-893e-b91a772cf6bb admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:14:20.174823 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f452d571-bff5-4d99-893e-b91a772cf6bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:14:20.233198 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f452d571-bff5-4d99-893e-b91a772cf6bb admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:14:20.236872 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd29558-d51c-4aa1-925b-d3a01314aa82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:14:20.238087 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd29558-d51c-4aa1-925b-d3a01314aa82 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:20.238369 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd29558-d51c-4aa1-925b-d3a01314aa82 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:20.238648 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd29558-d51c-4aa1-925b-d3a01314aa82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:20.247887 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd29558-d51c-4aa1-925b-d3a01314aa82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:14:20.249331 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd29558-d51c-4aa1-925b-d3a01314aa82 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:14:20.258917 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1322/2674] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:14:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:14:20.537521 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679f09a0-27d1-4334-8b60-56ab4babbbd7 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:14:20.539024 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-679f09a0-27d1-4334-8b60-56ab4babbbd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:14:20.554400 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679f09a0-27d1-4334-8b60-56ab4babbbd7 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:14:20.560383 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25040bb8-460f-491b-a729-4973d8d30d3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:14:20.561608 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25040bb8-460f-491b-a729-4973d8d30d3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:20.561787 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25040bb8-460f-491b-a729-4973d8d30d3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:20.562069 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25040bb8-460f-491b-a729-4973d8d30d3e admin admin] PATH_INFO: `/v3/projects/59d3e42c3b494b34b9eba6a3f682a632` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:20.563317 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25040bb8-460f-491b-a729-4973d8d30d3e admin admin] RBAC: Authorizing `identity:delete_project(project_id=59d3e42c3b494b34b9eba6a3f682a632)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:14:20.565354 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25040bb8-460f-491b-a729-4973d8d30d3e admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:14:20.613204 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-25040bb8-460f-491b-a729-4973d8d30d3e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59d3e42c3b494b34b9eba6a3f682a632', 'request_id': 'req-25040bb8-460f-491b-a729-4973d8d30d3e'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:20.619308 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-25040bb8-460f-491b-a729-4973d8d30d3e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59d3e42c3b494b34b9eba6a3f682a632', 'request_id': 'req-25040bb8-460f-491b-a729-4973d8d30d3e'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:20.621298 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:INSERT=2,keystone:SELECT=93,keystone:DELETE=5 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:14:20.625810 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-25040bb8-460f-491b-a729-4973d8d30d3e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59d3e42c3b494b34b9eba6a3f682a632', 'request_id': 'req-25040bb8-460f-491b-a729-4973d8d30d3e'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:20.639718 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-25040bb8-460f-491b-a729-4973d8d30d3e admin admin] The token cache is being invalidate because project 59d3e42c3b494b34b9eba6a3f682a632 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 16:14:20.639965 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-25040bb8-460f-491b-a729-4973d8d30d3e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:20.640568 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-25040bb8-460f-491b-a729-4973d8d30d3e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:20.642252 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1353/2675] 10.222.0.99 () {58 vars in 1232 bytes} [Thu Jul 14 16:14:20 2022] DELETE /identity/v3/projects/59d3e42c3b494b34b9eba6a3f682a632 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:14:30.254744 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=49,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:14:30.636287 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=4,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:14:38.350569 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd29558-d51c-4aa1-925b-d3a01314aa82 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:14:38.359437 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd29558-d51c-4aa1-925b-d3a01314aa82 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:14:38.410111 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52056a20-223e-4676-913d-1f799237ac93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:14:38.410772 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52056a20-223e-4676-913d-1f799237ac93 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:38.411061 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52056a20-223e-4676-913d-1f799237ac93 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:38.411345 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52056a20-223e-4676-913d-1f799237ac93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:38.413190 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-52056a20-223e-4676-913d-1f799237ac93 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:14:38.478013 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52056a20-223e-4676-913d-1f799237ac93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:14:38.479480 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52056a20-223e-4676-913d-1f799237ac93 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:14:38.489723 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1323/2676] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:14:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:14:42.048986 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25040bb8-460f-491b-a729-4973d8d30d3e admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:14:42.051198 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-25040bb8-460f-491b-a729-4973d8d30d3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:14:42.065905 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25040bb8-460f-491b-a729-4973d8d30d3e admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:14:42.072536 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7a6296-9fa1-4aa6-91f6-9a8140b5179a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:14:42.073432 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7a6296-9fa1-4aa6-91f6-9a8140b5179a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:42.073718 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7a6296-9fa1-4aa6-91f6-9a8140b5179a admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:42.074012 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7a6296-9fa1-4aa6-91f6-9a8140b5179a admin admin] PATH_INFO: `/v3/users/46f52ad195a34123982d238ea351172d` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:42.075356 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7a6296-9fa1-4aa6-91f6-9a8140b5179a admin admin] RBAC: Authorizing `identity:delete_user(user_id=46f52ad195a34123982d238ea351172d)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:14:42.077352 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7a6296-9fa1-4aa6-91f6-9a8140b5179a admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:14:42.135671 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-5d7a6296-9fa1-4aa6-91f6-9a8140b5179a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46f52ad195a34123982d238ea351172d'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:42.143751 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-5d7a6296-9fa1-4aa6-91f6-9a8140b5179a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46f52ad195a34123982d238ea351172d'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:42.157142 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-5d7a6296-9fa1-4aa6-91f6-9a8140b5179a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46f52ad195a34123982d238ea351172d'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:42.157823 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-5d7a6296-9fa1-4aa6-91f6-9a8140b5179a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46f52ad195a34123982d238ea351172d'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:42.158472 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-5d7a6296-9fa1-4aa6-91f6-9a8140b5179a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46f52ad195a34123982d238ea351172d'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:42.172024 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1354/2677] 10.222.0.99 () {58 vars in 1223 bytes} [Thu Jul 14 16:14:42 2022] DELETE /identity/v3/users/46f52ad195a34123982d238ea351172d => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 16:14:42.369697 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52056a20-223e-4676-913d-1f799237ac93 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:14:42.382184 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52056a20-223e-4676-913d-1f799237ac93 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:14:42.458078 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3eafd2-5c4e-47ef-955d-78a5541d3094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:14:42.459386 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3eafd2-5c4e-47ef-955d-78a5541d3094 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:42.459851 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3eafd2-5c4e-47ef-955d-78a5541d3094 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:42.460265 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3eafd2-5c4e-47ef-955d-78a5541d3094 admin admin] PATH_INFO: `/v3/projects/67ea0d1ef31b495b8dad23f39cedf1bf` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:42.462179 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3eafd2-5c4e-47ef-955d-78a5541d3094 admin admin] RBAC: Authorizing `identity:delete_project(project_id=67ea0d1ef31b495b8dad23f39cedf1bf)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:14:42.465030 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3eafd2-5c4e-47ef-955d-78a5541d3094 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:14:42.522473 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2f3eafd2-5c4e-47ef-955d-78a5541d3094 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67ea0d1ef31b495b8dad23f39cedf1bf', 'request_id': 'req-2f3eafd2-5c4e-47ef-955d-78a5541d3094'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:42.524409 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:14:42.529519 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2f3eafd2-5c4e-47ef-955d-78a5541d3094 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67ea0d1ef31b495b8dad23f39cedf1bf', 'request_id': 'req-2f3eafd2-5c4e-47ef-955d-78a5541d3094'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:42.536612 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2f3eafd2-5c4e-47ef-955d-78a5541d3094 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67ea0d1ef31b495b8dad23f39cedf1bf', 'request_id': 'req-2f3eafd2-5c4e-47ef-955d-78a5541d3094'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:42.552024 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2f3eafd2-5c4e-47ef-955d-78a5541d3094 admin admin] The token cache is being invalidate because project 67ea0d1ef31b495b8dad23f39cedf1bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 16:14:42.552315 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2f3eafd2-5c4e-47ef-955d-78a5541d3094 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:42.552969 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2f3eafd2-5c4e-47ef-955d-78a5541d3094 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:42.554539 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1324/2678] 10.222.0.99 () {58 vars in 1232 bytes} [Thu Jul 14 16:14:42 2022] DELETE /identity/v3/projects/67ea0d1ef31b495b8dad23f39cedf1bf => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:14:42.559540 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b231ae-1601-4dfd-aeb1-3028dabdddb5 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:42.559868 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b231ae-1601-4dfd-aeb1-3028dabdddb5 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:42.560149 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b231ae-1601-4dfd-aeb1-3028dabdddb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:42.578228 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-02b231ae-1601-4dfd-aeb1-3028dabdddb5 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:14:42.579503 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-02b231ae-1601-4dfd-aeb1-3028dabdddb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:14:42.647322 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1355/2679] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 16:14:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:14:42.655410 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e43f4a-ec6e-47e0-aa90-1f4758c4e3f3 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:42.655746 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e43f4a-ec6e-47e0-aa90-1f4758c4e3f3 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:42.656023 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e43f4a-ec6e-47e0-aa90-1f4758c4e3f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:42.675825 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-23e43f4a-ec6e-47e0-aa90-1f4758c4e3f3 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:14:42.677156 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-23e43f4a-ec6e-47e0-aa90-1f4758c4e3f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:14:42.685226 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1325/2680] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 16:14:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:14:42.689990 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b231ae-1601-4dfd-aeb1-3028dabdddb5 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:14:42.702798 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b231ae-1601-4dfd-aeb1-3028dabdddb5 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:14:42.707840 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c83c437-fec1-4f31-82ef-2c5e9d9a31e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:14:42.708516 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c83c437-fec1-4f31-82ef-2c5e9d9a31e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:42.708801 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c83c437-fec1-4f31-82ef-2c5e9d9a31e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:42.709077 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c83c437-fec1-4f31-82ef-2c5e9d9a31e6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:42.709820 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c83c437-fec1-4f31-82ef-2c5e9d9a31e6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 16:14:42.710128 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c83c437-fec1-4f31-82ef-2c5e9d9a31e6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:14:42.711785 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c83c437-fec1-4f31-82ef-2c5e9d9a31e6 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:14:42.739783 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1356/2681] 10.222.0.99 () {58 vars in 1152 bytes} [Thu Jul 14 16:14:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 16:14:42.744912 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e43f4a-ec6e-47e0-aa90-1f4758c4e3f3 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:14:42.756136 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e43f4a-ec6e-47e0-aa90-1f4758c4e3f3 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:14:42.761090 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5b5ccb-b505-4c24-8bbb-3c66a212fcf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:14:42.761784 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5b5ccb-b505-4c24-8bbb-3c66a212fcf5 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:42.762075 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5b5ccb-b505-4c24-8bbb-3c66a212fcf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:42.762359 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5b5ccb-b505-4c24-8bbb-3c66a212fcf5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:42.763081 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5b5ccb-b505-4c24-8bbb-3c66a212fcf5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:14:42.765055 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5b5ccb-b505-4c24-8bbb-3c66a212fcf5 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:14:42.797673 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1326/2682] 10.222.0.99 () {60 vars in 1152 bytes} [Thu Jul 14 16:14:42 2022] POST /identity/v3/projects => generated 385 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 16:14:42.800642 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c83c437-fec1-4f31-82ef-2c5e9d9a31e6 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:14:42.813033 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c83c437-fec1-4f31-82ef-2c5e9d9a31e6 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:14:42.881367 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f233ced-489c-496e-b989-c44465d2ce51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:14:42.881966 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f233ced-489c-496e-b989-c44465d2ce51 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:42.882250 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f233ced-489c-496e-b989-c44465d2ce51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:42.882532 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f233ced-489c-496e-b989-c44465d2ce51 admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:42.883235 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f233ced-489c-496e-b989-c44465d2ce51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:14:42.885203 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f233ced-489c-496e-b989-c44465d2ce51 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:14:42.891935 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:14:42.912024 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1357/2683] 10.222.0.99 () {60 vars in 1143 bytes} [Thu Jul 14 16:14:42 2022] POST /identity/v3/users => generated 293 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 16:14:42.916497 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5b5ccb-b505-4c24-8bbb-3c66a212fcf5 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:14:42.927865 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5b5ccb-b505-4c24-8bbb-3c66a212fcf5 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:14:42.932864 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead3ddc2-eece-4517-9dc0-9f664265da30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:14:42.933546 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead3ddc2-eece-4517-9dc0-9f664265da30 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:42.933831 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead3ddc2-eece-4517-9dc0-9f664265da30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:42.934114 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead3ddc2-eece-4517-9dc0-9f664265da30 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:42.934859 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead3ddc2-eece-4517-9dc0-9f664265da30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:14:42.936520 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead3ddc2-eece-4517-9dc0-9f664265da30 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:14:42.952470 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1327/2684] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 16:14:42 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 16:14:42.957511 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f233ced-489c-496e-b989-c44465d2ce51 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:14:42.968252 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f233ced-489c-496e-b989-c44465d2ce51 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:14:42.975074 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1ac3b8-b30d-4973-bcc9-5702778e90e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:14:42.975776 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1ac3b8-b30d-4973-bcc9-5702778e90e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:42.976163 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1ac3b8-b30d-4973-bcc9-5702778e90e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:42.976495 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1ac3b8-b30d-4973-bcc9-5702778e90e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:42.977275 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1ac3b8-b30d-4973-bcc9-5702778e90e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:14:42.978896 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1ac3b8-b30d-4973-bcc9-5702778e90e5 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:14:42.992908 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1358/2685] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 16:14:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 16:14:42.994299 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead3ddc2-eece-4517-9dc0-9f664265da30 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:14:43.004514 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead3ddc2-eece-4517-9dc0-9f664265da30 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:14:43.009542 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28db62b0-6b7a-4c49-934e-d710ccf229c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:14:43.010425 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28db62b0-6b7a-4c49-934e-d710ccf229c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:43.010710 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28db62b0-6b7a-4c49-934e-d710ccf229c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:43.010986 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28db62b0-6b7a-4c49-934e-d710ccf229c4 admin admin] PATH_INFO: `/v3/projects/9dd0ea5816ca4f8c9d58a902ff448b55/users/1f335c7f66b84c259e002f69c018266d/roles/b8c98fddc0fd4436b577fd1d3361bc38` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:43.026157 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28db62b0-6b7a-4c49-934e-d710ccf229c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9dd0ea5816ca4f8c9d58a902ff448b55, user_id=1f335c7f66b84c259e002f69c018266d, role_id=b8c98fddc0fd4436b577fd1d3361bc38)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:14:43.028844 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28db62b0-6b7a-4c49-934e-d710ccf229c4 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:14:43.043416 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1328/2686] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 16:14:42 2022] PUT /identity/v3/projects/9dd0ea5816ca4f8c9d58a902ff448b55/users/1f335c7f66b84c259e002f69c018266d/roles/b8c98fddc0fd4436b577fd1d3361bc38 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:14:43.054922 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c96e819-0ef0-47b7-9134-63ab8db9da74 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:43.055508 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c96e819-0ef0-47b7-9134-63ab8db9da74 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:43.055508 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c96e819-0ef0-47b7-9134-63ab8db9da74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:43.088671 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-9c96e819-0ef0-47b7-9134-63ab8db9da74 None None] MFA Rules not processed for user `1f335c7f66b84c259e002f69c018266d`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:14:43.089970 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-9c96e819-0ef0-47b7-9134-63ab8db9da74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:14:43.172814 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1359/2687] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 16:14:43 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:14:43.178028 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28db62b0-6b7a-4c49-934e-d710ccf229c4 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:14:43.243423 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28db62b0-6b7a-4c49-934e-d710ccf229c4 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:14:43.249034 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080e9c15-160a-49b1-b505-ff31d5c6516a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:14:43.250199 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080e9c15-160a-49b1-b505-ff31d5c6516a admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:43.250199 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080e9c15-160a-49b1-b505-ff31d5c6516a admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:43.251318 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080e9c15-160a-49b1-b505-ff31d5c6516a admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:43.251318 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080e9c15-160a-49b1-b505-ff31d5c6516a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:14:43.254108 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080e9c15-160a-49b1-b505-ff31d5c6516a admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:14:43.283310 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1329/2688] 10.222.0.99 () {60 vars in 1152 bytes} [Thu Jul 14 16:14:43 2022] POST /identity/v3/projects => generated 385 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 16:14:43.287481 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c96e819-0ef0-47b7-9134-63ab8db9da74 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:14:43.299361 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c96e819-0ef0-47b7-9134-63ab8db9da74 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:14:43.362470 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a85d3f4-2a31-4046-beb9-de18e7df2956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:14:43.363159 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a85d3f4-2a31-4046-beb9-de18e7df2956 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:43.363445 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a85d3f4-2a31-4046-beb9-de18e7df2956 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:43.363759 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a85d3f4-2a31-4046-beb9-de18e7df2956 admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:43.364498 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a85d3f4-2a31-4046-beb9-de18e7df2956 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:14:43.366422 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a85d3f4-2a31-4046-beb9-de18e7df2956 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:14:43.389472 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1360/2689] 10.222.0.99 () {60 vars in 1143 bytes} [Thu Jul 14 16:14:43 2022] POST /identity/v3/users => generated 299 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 16:14:43.394649 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080e9c15-160a-49b1-b505-ff31d5c6516a admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:14:43.405439 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080e9c15-160a-49b1-b505-ff31d5c6516a admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:14:43.410419 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5a0187-4202-4855-8472-c3eb0f9ce9ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:14:43.411093 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5a0187-4202-4855-8472-c3eb0f9ce9ac admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:43.411386 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5a0187-4202-4855-8472-c3eb0f9ce9ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:43.411714 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5a0187-4202-4855-8472-c3eb0f9ce9ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:43.412440 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5a0187-4202-4855-8472-c3eb0f9ce9ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:14:43.414062 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5a0187-4202-4855-8472-c3eb0f9ce9ac admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:14:43.428754 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1330/2690] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 16:14:43 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:14:43.435012 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a85d3f4-2a31-4046-beb9-de18e7df2956 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:14:43.446102 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a85d3f4-2a31-4046-beb9-de18e7df2956 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:14:43.450834 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec3d52b-d4e5-420d-a66c-0f0e303cc8d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:14:43.451760 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec3d52b-d4e5-420d-a66c-0f0e303cc8d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:43.452052 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec3d52b-d4e5-420d-a66c-0f0e303cc8d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:43.452339 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec3d52b-d4e5-420d-a66c-0f0e303cc8d2 admin admin] PATH_INFO: `/v3/projects/f1ea57ec6da04094adc70488d201344e/users/becfbad1553f4bd09a031d2315c8fee7/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:43.468043 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec3d52b-d4e5-420d-a66c-0f0e303cc8d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1ea57ec6da04094adc70488d201344e, user_id=becfbad1553f4bd09a031d2315c8fee7, role_id=d0f6ba1314f14d9f9c816bdb5d0c3aac)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:14:43.470696 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec3d52b-d4e5-420d-a66c-0f0e303cc8d2 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:14:43.483918 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1361/2691] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 16:14:43 2022] PUT /identity/v3/projects/f1ea57ec6da04094adc70488d201344e/users/becfbad1553f4bd09a031d2315c8fee7/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:14:43.488074 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5a0187-4202-4855-8472-c3eb0f9ce9ac admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:14:43.499193 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5a0187-4202-4855-8472-c3eb0f9ce9ac admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:14:43.547751 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:14:43.559257 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b748e8-3398-45da-821c-ce223258c26a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:14:43.559991 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b748e8-3398-45da-821c-ce223258c26a admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:43.560284 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b748e8-3398-45da-821c-ce223258c26a admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:43.560567 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b748e8-3398-45da-821c-ce223258c26a admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:43.561302 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b748e8-3398-45da-821c-ce223258c26a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:14:43.562916 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b748e8-3398-45da-821c-ce223258c26a admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:14:43.573399 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1331/2692] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 16:14:43 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:14:43.577726 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec3d52b-d4e5-420d-a66c-0f0e303cc8d2 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:14:43.589363 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec3d52b-d4e5-420d-a66c-0f0e303cc8d2 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:14:43.594008 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54baf12-e287-4530-9635-249acce54cec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:14:43.594896 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54baf12-e287-4530-9635-249acce54cec admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:43.595185 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54baf12-e287-4530-9635-249acce54cec admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:43.595467 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54baf12-e287-4530-9635-249acce54cec admin admin] PATH_INFO: `/v3/projects/f1ea57ec6da04094adc70488d201344e/users/becfbad1553f4bd09a031d2315c8fee7/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:43.598116 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54baf12-e287-4530-9635-249acce54cec admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1ea57ec6da04094adc70488d201344e, user_id=becfbad1553f4bd09a031d2315c8fee7, role_id=d0f6ba1314f14d9f9c816bdb5d0c3aac)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:14:43.600797 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54baf12-e287-4530-9635-249acce54cec admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:14:43.611030 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1362/2693] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 16:14:43 2022] PUT /identity/v3/projects/f1ea57ec6da04094adc70488d201344e/users/becfbad1553f4bd09a031d2315c8fee7/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:14:43.623377 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf290fd7-fbd2-45a9-bc5b-7d68c23ba044 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:43.623706 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf290fd7-fbd2-45a9-bc5b-7d68c23ba044 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:43.623987 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf290fd7-fbd2-45a9-bc5b-7d68c23ba044 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:43.650822 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-bf290fd7-fbd2-45a9-bc5b-7d68c23ba044 None None] MFA Rules not processed for user `becfbad1553f4bd09a031d2315c8fee7`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:14:43.652012 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-bf290fd7-fbd2-45a9-bc5b-7d68c23ba044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:14:43.718881 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1332/2694] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 16:14:43 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:14:43.730315 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54baf12-e287-4530-9635-249acce54cec admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:14:43.732128 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-d54baf12-e287-4530-9635-249acce54cec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:14:43.757052 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:14:43.795810 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54baf12-e287-4530-9635-249acce54cec admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:14:43.800270 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483a53fe-5f7b-4d89-9ecb-ddf22cfa35d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3e1a2b83f3d4d88ba5084dfaffd22f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:14:43.801128 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483a53fe-5f7b-4d89-9ecb-ddf22cfa35d2 service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:43.801420 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483a53fe-5f7b-4d89-9ecb-ddf22cfa35d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:43.801700 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483a53fe-5f7b-4d89-9ecb-ddf22cfa35d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:43.811155 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483a53fe-5f7b-4d89-9ecb-ddf22cfa35d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:14:43.812633 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483a53fe-5f7b-4d89-9ecb-ddf22cfa35d2 service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:14:43.823212 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1363/2695] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:14:43 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:14:43.850424 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf290fd7-fbd2-45a9-bc5b-7d68c23ba044 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:14:43.912850 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf290fd7-fbd2-45a9-bc5b-7d68c23ba044 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:14:43.917504 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf1e60c-5ea6-446c-9457-1b7f7d1d14b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:14:43.918375 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf1e60c-5ea6-446c-9457-1b7f7d1d14b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:43.918664 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf1e60c-5ea6-446c-9457-1b7f7d1d14b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:43.918946 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf1e60c-5ea6-446c-9457-1b7f7d1d14b1 admin admin] PATH_INFO: `/v3/users/1f335c7f66b84c259e002f69c018266d` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:43.920179 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf1e60c-5ea6-446c-9457-1b7f7d1d14b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f335c7f66b84c259e002f69c018266d)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:14:43.922525 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf1e60c-5ea6-446c-9457-1b7f7d1d14b1 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:14:43.973803 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-acf1e60c-5ea6-446c-9457-1b7f7d1d14b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f335c7f66b84c259e002f69c018266d'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:43.982180 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-acf1e60c-5ea6-446c-9457-1b7f7d1d14b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f335c7f66b84c259e002f69c018266d'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:43.995755 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-acf1e60c-5ea6-446c-9457-1b7f7d1d14b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f335c7f66b84c259e002f69c018266d'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:43.996496 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-acf1e60c-5ea6-446c-9457-1b7f7d1d14b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f335c7f66b84c259e002f69c018266d'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:43.997124 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-acf1e60c-5ea6-446c-9457-1b7f7d1d14b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f335c7f66b84c259e002f69c018266d'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:43.999276 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:14:44.012560 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1333/2696] 10.222.0.99 () {58 vars in 1223 bytes} [Thu Jul 14 16:14:43 2022] DELETE /identity/v3/users/1f335c7f66b84c259e002f69c018266d => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:14:44.017104 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bac2aa-2b9b-474e-b3a5-3d16244e0550 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:44.017405 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bac2aa-2b9b-474e-b3a5-3d16244e0550 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:44.017692 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bac2aa-2b9b-474e-b3a5-3d16244e0550 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:44.036092 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-10bac2aa-2b9b-474e-b3a5-3d16244e0550 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:14:44.037303 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-10bac2aa-2b9b-474e-b3a5-3d16244e0550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:14:44.095996 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1364/2697] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 16:14:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:14:44.106270 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf1e60c-5ea6-446c-9457-1b7f7d1d14b1 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:14:44.107782 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-acf1e60c-5ea6-446c-9457-1b7f7d1d14b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:14:44.166598 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf1e60c-5ea6-446c-9457-1b7f7d1d14b1 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:14:44.170343 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cc7739-9716-4d74-a0a7-bae1dd237098 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:14:44.170923 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cc7739-9716-4d74-a0a7-bae1dd237098 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:44.171212 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cc7739-9716-4d74-a0a7-bae1dd237098 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:44.171499 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cc7739-9716-4d74-a0a7-bae1dd237098 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:44.181977 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cc7739-9716-4d74-a0a7-bae1dd237098 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:14:44.183425 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cc7739-9716-4d74-a0a7-bae1dd237098 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:14:44.193214 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1334/2698] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:14:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 16:14:44.476055 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bac2aa-2b9b-474e-b3a5-3d16244e0550 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:14:44.477508 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-10bac2aa-2b9b-474e-b3a5-3d16244e0550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:14:44.492383 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bac2aa-2b9b-474e-b3a5-3d16244e0550 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:14:44.497998 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a98d832-17df-4a17-8176-f238c3ec5d34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:14:44.498904 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a98d832-17df-4a17-8176-f238c3ec5d34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:44.499189 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a98d832-17df-4a17-8176-f238c3ec5d34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:44.499486 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a98d832-17df-4a17-8176-f238c3ec5d34 admin admin] PATH_INFO: `/v3/projects/9dd0ea5816ca4f8c9d58a902ff448b55` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:44.501307 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a98d832-17df-4a17-8176-f238c3ec5d34 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9dd0ea5816ca4f8c9d58a902ff448b55)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:14:44.504203 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a98d832-17df-4a17-8176-f238c3ec5d34 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:14:44.554063 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-2a98d832-17df-4a17-8176-f238c3ec5d34 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9dd0ea5816ca4f8c9d58a902ff448b55', 'request_id': 'req-2a98d832-17df-4a17-8176-f238c3ec5d34'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:44.560302 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-2a98d832-17df-4a17-8176-f238c3ec5d34 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9dd0ea5816ca4f8c9d58a902ff448b55', 'request_id': 'req-2a98d832-17df-4a17-8176-f238c3ec5d34'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:44.567399 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-2a98d832-17df-4a17-8176-f238c3ec5d34 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9dd0ea5816ca4f8c9d58a902ff448b55', 'request_id': 'req-2a98d832-17df-4a17-8176-f238c3ec5d34'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:44.581460 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-2a98d832-17df-4a17-8176-f238c3ec5d34 admin admin] The token cache is being invalidate because project 9dd0ea5816ca4f8c9d58a902ff448b55 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 16:14:44.581755 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-2a98d832-17df-4a17-8176-f238c3ec5d34 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:44.582382 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-2a98d832-17df-4a17-8176-f238c3ec5d34 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:44.584408 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1365/2699] 10.222.0.99 () {58 vars in 1232 bytes} [Thu Jul 14 16:14:44 2022] DELETE /identity/v3/projects/9dd0ea5816ca4f8c9d58a902ff448b55 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:14:44.588173 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cc7739-9716-4d74-a0a7-bae1dd237098 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:14:44.598158 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cc7739-9716-4d74-a0a7-bae1dd237098 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:14:44.660861 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297a2007-a025-4660-bf5c-4658ee7f2c66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:14:44.661766 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297a2007-a025-4660-bf5c-4658ee7f2c66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:44.662051 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297a2007-a025-4660-bf5c-4658ee7f2c66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:44.662343 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297a2007-a025-4660-bf5c-4658ee7f2c66 admin admin] PATH_INFO: `/v3/users/becfbad1553f4bd09a031d2315c8fee7` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:44.663528 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297a2007-a025-4660-bf5c-4658ee7f2c66 admin admin] RBAC: Authorizing `identity:delete_user(user_id=becfbad1553f4bd09a031d2315c8fee7)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:14:44.665464 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297a2007-a025-4660-bf5c-4658ee7f2c66 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:14:44.713550 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-297a2007-a025-4660-bf5c-4658ee7f2c66 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'becfbad1553f4bd09a031d2315c8fee7'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:44.720590 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-297a2007-a025-4660-bf5c-4658ee7f2c66 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'becfbad1553f4bd09a031d2315c8fee7'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:44.733476 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-297a2007-a025-4660-bf5c-4658ee7f2c66 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'becfbad1553f4bd09a031d2315c8fee7'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:44.734129 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-297a2007-a025-4660-bf5c-4658ee7f2c66 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'becfbad1553f4bd09a031d2315c8fee7'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:44.734737 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-297a2007-a025-4660-bf5c-4658ee7f2c66 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'becfbad1553f4bd09a031d2315c8fee7'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:44.747559 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1335/2700] 10.222.0.99 () {58 vars in 1223 bytes} [Thu Jul 14 16:14:44 2022] DELETE /identity/v3/users/becfbad1553f4bd09a031d2315c8fee7 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:14:45.026479 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a98d832-17df-4a17-8176-f238c3ec5d34 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:14:45.028159 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-2a98d832-17df-4a17-8176-f238c3ec5d34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:14:45.046261 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a98d832-17df-4a17-8176-f238c3ec5d34 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:14:45.091311 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:14:45.107251 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2fa153-f44e-4cfc-b95b-b4750dc2ad55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:14:45.108196 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2fa153-f44e-4cfc-b95b-b4750dc2ad55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:45.108480 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2fa153-f44e-4cfc-b95b-b4750dc2ad55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:45.108761 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2fa153-f44e-4cfc-b95b-b4750dc2ad55 admin admin] PATH_INFO: `/v3/projects/f1ea57ec6da04094adc70488d201344e` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:45.109978 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2fa153-f44e-4cfc-b95b-b4750dc2ad55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1ea57ec6da04094adc70488d201344e)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:14:45.113206 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2fa153-f44e-4cfc-b95b-b4750dc2ad55 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:14:45.164264 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-1b2fa153-f44e-4cfc-b95b-b4750dc2ad55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1ea57ec6da04094adc70488d201344e', 'request_id': 'req-1b2fa153-f44e-4cfc-b95b-b4750dc2ad55'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:45.170897 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-1b2fa153-f44e-4cfc-b95b-b4750dc2ad55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1ea57ec6da04094adc70488d201344e', 'request_id': 'req-1b2fa153-f44e-4cfc-b95b-b4750dc2ad55'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:45.177749 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-1b2fa153-f44e-4cfc-b95b-b4750dc2ad55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1ea57ec6da04094adc70488d201344e', 'request_id': 'req-1b2fa153-f44e-4cfc-b95b-b4750dc2ad55'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:45.191275 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-1b2fa153-f44e-4cfc-b95b-b4750dc2ad55 admin admin] The token cache is being invalidate because project f1ea57ec6da04094adc70488d201344e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 16:14:45.191561 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-1b2fa153-f44e-4cfc-b95b-b4750dc2ad55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:45.192227 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-1b2fa153-f44e-4cfc-b95b-b4750dc2ad55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:45.193770 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1366/2701] 10.222.0.99 () {58 vars in 1232 bytes} [Thu Jul 14 16:14:45 2022] DELETE /identity/v3/projects/f1ea57ec6da04094adc70488d201344e => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:14:45.198669 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7951388c-c1d2-4d63-b083-ebae6408c342 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:45.198968 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7951388c-c1d2-4d63-b083-ebae6408c342 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:45.199284 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7951388c-c1d2-4d63-b083-ebae6408c342 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:45.217552 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-7951388c-c1d2-4d63-b083-ebae6408c342 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:14:45.218824 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-7951388c-c1d2-4d63-b083-ebae6408c342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:14:45.227869 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:14:45.284163 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1336/2702] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 16:14:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:14:45.291831 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c50a920-bdc2-4887-9d52-f727321824c8 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:45.292124 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c50a920-bdc2-4887-9d52-f727321824c8 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:45.292406 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c50a920-bdc2-4887-9d52-f727321824c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:45.311193 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-1c50a920-bdc2-4887-9d52-f727321824c8 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:14:45.312446 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-1c50a920-bdc2-4887-9d52-f727321824c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:14:45.321255 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1367/2703] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 16:14:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:14:45.325444 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7951388c-c1d2-4d63-b083-ebae6408c342 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:14:45.335673 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7951388c-c1d2-4d63-b083-ebae6408c342 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:14:45.340307 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d383dc2-ab19-4329-aab3-8dc7dddb1427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:14:45.340972 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d383dc2-ab19-4329-aab3-8dc7dddb1427 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:45.341318 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d383dc2-ab19-4329-aab3-8dc7dddb1427 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:45.341602 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d383dc2-ab19-4329-aab3-8dc7dddb1427 admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:45.342367 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d383dc2-ab19-4329-aab3-8dc7dddb1427 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 16:14:45.342681 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d383dc2-ab19-4329-aab3-8dc7dddb1427 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:14:45.344356 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d383dc2-ab19-4329-aab3-8dc7dddb1427 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:14:45.355978 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1337/2704] 10.222.0.99 () {58 vars in 1152 bytes} [Thu Jul 14 16:14:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 16:14:45.360055 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c50a920-bdc2-4887-9d52-f727321824c8 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:14:45.369788 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c50a920-bdc2-4887-9d52-f727321824c8 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:14:45.374363 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09452acd-3865-4354-8922-4192d9fe4e75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:14:45.375001 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09452acd-3865-4354-8922-4192d9fe4e75 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:45.375284 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09452acd-3865-4354-8922-4192d9fe4e75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:45.375562 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09452acd-3865-4354-8922-4192d9fe4e75 admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:45.376275 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09452acd-3865-4354-8922-4192d9fe4e75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:14:45.378882 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09452acd-3865-4354-8922-4192d9fe4e75 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:14:45.404847 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1368/2705] 10.222.0.99 () {60 vars in 1152 bytes} [Thu Jul 14 16:14:45 2022] POST /identity/v3/projects => generated 381 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 16:14:45.409475 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d383dc2-ab19-4329-aab3-8dc7dddb1427 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:14:45.419429 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d383dc2-ab19-4329-aab3-8dc7dddb1427 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:14:45.476599 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4459273-3364-43ca-a0b5-4c0126709cf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:14:45.477187 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4459273-3364-43ca-a0b5-4c0126709cf7 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:45.477477 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4459273-3364-43ca-a0b5-4c0126709cf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:45.477757 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4459273-3364-43ca-a0b5-4c0126709cf7 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:45.478453 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4459273-3364-43ca-a0b5-4c0126709cf7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:14:45.480410 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4459273-3364-43ca-a0b5-4c0126709cf7 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:14:45.501458 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1338/2706] 10.222.0.99 () {60 vars in 1143 bytes} [Thu Jul 14 16:14:45 2022] POST /identity/v3/users => generated 291 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 16:14:45.505682 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09452acd-3865-4354-8922-4192d9fe4e75 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:14:45.516374 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09452acd-3865-4354-8922-4192d9fe4e75 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:14:45.521317 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d730cf3-a8eb-41e6-b982-546901272969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:14:45.522012 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d730cf3-a8eb-41e6-b982-546901272969 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:45.522289 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d730cf3-a8eb-41e6-b982-546901272969 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:45.522574 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d730cf3-a8eb-41e6-b982-546901272969 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:45.523310 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d730cf3-a8eb-41e6-b982-546901272969 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:14:45.524984 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d730cf3-a8eb-41e6-b982-546901272969 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:14:45.536353 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1369/2707] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 16:14:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:14:45.540458 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4459273-3364-43ca-a0b5-4c0126709cf7 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:14:45.551673 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4459273-3364-43ca-a0b5-4c0126709cf7 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:14:45.556529 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432fb2f5-9e71-45c2-890e-da6bd6c91cc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:14:45.557187 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432fb2f5-9e71-45c2-890e-da6bd6c91cc9 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:45.557460 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432fb2f5-9e71-45c2-890e-da6bd6c91cc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:45.557738 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432fb2f5-9e71-45c2-890e-da6bd6c91cc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:45.558496 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432fb2f5-9e71-45c2-890e-da6bd6c91cc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:14:45.560150 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432fb2f5-9e71-45c2-890e-da6bd6c91cc9 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:14:45.570361 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1339/2708] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 16:14:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:14:45.574494 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d730cf3-a8eb-41e6-b982-546901272969 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:14:45.584421 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d730cf3-a8eb-41e6-b982-546901272969 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:14:45.589045 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a811dfe-87e9-4a76-8401-150655d3d1e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:14:45.589917 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a811dfe-87e9-4a76-8401-150655d3d1e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:45.590200 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a811dfe-87e9-4a76-8401-150655d3d1e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:45.590474 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a811dfe-87e9-4a76-8401-150655d3d1e9 admin admin] PATH_INFO: `/v3/projects/040d9931712242f486545152296b77a0/users/67bbe050db15416a919eff62d1573f63/roles/b8c98fddc0fd4436b577fd1d3361bc38` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:45.605494 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a811dfe-87e9-4a76-8401-150655d3d1e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=040d9931712242f486545152296b77a0, user_id=67bbe050db15416a919eff62d1573f63, role_id=b8c98fddc0fd4436b577fd1d3361bc38)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:14:45.608228 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a811dfe-87e9-4a76-8401-150655d3d1e9 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:14:45.622599 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1370/2709] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 16:14:45 2022] PUT /identity/v3/projects/040d9931712242f486545152296b77a0/users/67bbe050db15416a919eff62d1573f63/roles/b8c98fddc0fd4436b577fd1d3361bc38 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:14:45.633162 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bffacf2-1211-4913-84a4-09b78dd4cc75 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:45.633685 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bffacf2-1211-4913-84a4-09b78dd4cc75 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:45.633797 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bffacf2-1211-4913-84a4-09b78dd4cc75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:45.661079 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-1bffacf2-1211-4913-84a4-09b78dd4cc75 None None] MFA Rules not processed for user `67bbe050db15416a919eff62d1573f63`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:14:45.662303 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-1bffacf2-1211-4913-84a4-09b78dd4cc75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:14:45.727190 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1340/2710] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 16:14:45 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:14:45.731758 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a811dfe-87e9-4a76-8401-150655d3d1e9 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:14:45.794947 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a811dfe-87e9-4a76-8401-150655d3d1e9 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:14:45.800507 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1acabd9-cbb5-4000-a201-5cf28ed0e681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:14:45.801170 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1acabd9-cbb5-4000-a201-5cf28ed0e681 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:45.801457 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1acabd9-cbb5-4000-a201-5cf28ed0e681 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:45.801737 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1acabd9-cbb5-4000-a201-5cf28ed0e681 admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:45.802437 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1acabd9-cbb5-4000-a201-5cf28ed0e681 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:14:45.804512 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1acabd9-cbb5-4000-a201-5cf28ed0e681 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:14:45.827954 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:14:45.840891 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1371/2711] 10.222.0.99 () {60 vars in 1152 bytes} [Thu Jul 14 16:14:45 2022] POST /identity/v3/projects => generated 381 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 16:14:45.841789 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bffacf2-1211-4913-84a4-09b78dd4cc75 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:14:45.853697 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bffacf2-1211-4913-84a4-09b78dd4cc75 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:14:45.858319 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:14:45.913412 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12dc921-276b-4a68-9fcd-2a29d63fa777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:14:45.914124 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12dc921-276b-4a68-9fcd-2a29d63fa777 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:45.914408 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12dc921-276b-4a68-9fcd-2a29d63fa777 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:45.914689 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12dc921-276b-4a68-9fcd-2a29d63fa777 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:45.915402 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12dc921-276b-4a68-9fcd-2a29d63fa777 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:14:45.917338 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12dc921-276b-4a68-9fcd-2a29d63fa777 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:14:45.938107 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1341/2712] 10.222.0.99 () {60 vars in 1143 bytes} [Thu Jul 14 16:14:45 2022] POST /identity/v3/users => generated 297 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 16:14:45.943278 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1acabd9-cbb5-4000-a201-5cf28ed0e681 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:14:45.953563 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1acabd9-cbb5-4000-a201-5cf28ed0e681 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:14:45.958347 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cfa2e8-f59e-48af-8231-57a50c059ce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:14:45.959002 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cfa2e8-f59e-48af-8231-57a50c059ce1 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:45.959286 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cfa2e8-f59e-48af-8231-57a50c059ce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:45.959561 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cfa2e8-f59e-48af-8231-57a50c059ce1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:45.960322 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cfa2e8-f59e-48af-8231-57a50c059ce1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:14:45.961994 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cfa2e8-f59e-48af-8231-57a50c059ce1 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:14:45.973300 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1372/2713] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 16:14:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:14:45.977948 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12dc921-276b-4a68-9fcd-2a29d63fa777 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:14:45.989657 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12dc921-276b-4a68-9fcd-2a29d63fa777 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:14:45.995085 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad847da7-e52f-4037-b500-02327245cb2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:14:45.996057 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad847da7-e52f-4037-b500-02327245cb2f admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:45.996353 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad847da7-e52f-4037-b500-02327245cb2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:45.997140 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad847da7-e52f-4037-b500-02327245cb2f admin admin] PATH_INFO: `/v3/projects/7fdda46bd1e2466c9d5a4b1d42c45cc3/users/e268b9e49863439bac015fbee060fdf3/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:46.013023 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad847da7-e52f-4037-b500-02327245cb2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7fdda46bd1e2466c9d5a4b1d42c45cc3, user_id=e268b9e49863439bac015fbee060fdf3, role_id=d0f6ba1314f14d9f9c816bdb5d0c3aac)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:14:46.015711 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad847da7-e52f-4037-b500-02327245cb2f admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:14:46.028604 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1342/2714] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 16:14:45 2022] PUT /identity/v3/projects/7fdda46bd1e2466c9d5a4b1d42c45cc3/users/e268b9e49863439bac015fbee060fdf3/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:14:46.032561 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cfa2e8-f59e-48af-8231-57a50c059ce1 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:14:46.042817 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cfa2e8-f59e-48af-8231-57a50c059ce1 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:14:46.100025 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7c2019-b9ce-47e3-8ecf-e90d16f63b4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:14:46.100689 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7c2019-b9ce-47e3-8ecf-e90d16f63b4e admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:46.100975 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7c2019-b9ce-47e3-8ecf-e90d16f63b4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:46.101282 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7c2019-b9ce-47e3-8ecf-e90d16f63b4e admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:46.101978 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7c2019-b9ce-47e3-8ecf-e90d16f63b4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:14:46.103672 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7c2019-b9ce-47e3-8ecf-e90d16f63b4e admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:14:46.113903 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1373/2715] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 16:14:46 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:14:46.118288 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad847da7-e52f-4037-b500-02327245cb2f admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:14:46.129140 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad847da7-e52f-4037-b500-02327245cb2f admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:14:46.133736 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371daa45-30f1-41ce-922f-bd9f7eb2243d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:14:46.134615 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371daa45-30f1-41ce-922f-bd9f7eb2243d admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:46.134902 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371daa45-30f1-41ce-922f-bd9f7eb2243d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:46.135180 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371daa45-30f1-41ce-922f-bd9f7eb2243d admin admin] PATH_INFO: `/v3/projects/7fdda46bd1e2466c9d5a4b1d42c45cc3/users/e268b9e49863439bac015fbee060fdf3/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:46.137198 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371daa45-30f1-41ce-922f-bd9f7eb2243d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7fdda46bd1e2466c9d5a4b1d42c45cc3, user_id=e268b9e49863439bac015fbee060fdf3, role_id=d0f6ba1314f14d9f9c816bdb5d0c3aac)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:14:46.139871 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371daa45-30f1-41ce-922f-bd9f7eb2243d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:14:46.151668 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1343/2716] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 16:14:46 2022] PUT /identity/v3/projects/7fdda46bd1e2466c9d5a4b1d42c45cc3/users/e268b9e49863439bac015fbee060fdf3/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 16:14:46.165313 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2012c1c8-a287-4d48-a4bb-faa1b4c331d0 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:46.165614 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2012c1c8-a287-4d48-a4bb-faa1b4c331d0 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:46.165903 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2012c1c8-a287-4d48-a4bb-faa1b4c331d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:46.199243 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-2012c1c8-a287-4d48-a4bb-faa1b4c331d0 None None] MFA Rules not processed for user `e268b9e49863439bac015fbee060fdf3`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:14:46.201096 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-2012c1c8-a287-4d48-a4bb-faa1b4c331d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:14:46.273151 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1374/2717] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 16:14:46 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:14:46.284331 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371daa45-30f1-41ce-922f-bd9f7eb2243d admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:14:46.286083 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-371daa45-30f1-41ce-922f-bd9f7eb2243d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:14:46.348581 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371daa45-30f1-41ce-922f-bd9f7eb2243d admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:14:46.352703 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dffa5c-59d7-4e40-9748-3ecaea112465 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3e1a2b83f3d4d88ba5084dfaffd22f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:14:46.353358 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dffa5c-59d7-4e40-9748-3ecaea112465 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:46.353647 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dffa5c-59d7-4e40-9748-3ecaea112465 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:46.353930 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dffa5c-59d7-4e40-9748-3ecaea112465 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:46.363657 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dffa5c-59d7-4e40-9748-3ecaea112465 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:14:46.365133 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dffa5c-59d7-4e40-9748-3ecaea112465 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:14:46.374715 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1344/2718] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:14:46 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:14:46.471857 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2012c1c8-a287-4d48-a4bb-faa1b4c331d0 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:14:46.482690 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2012c1c8-a287-4d48-a4bb-faa1b4c331d0 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:14:46.486357 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44476505-e6a1-4b46-8cf3-c9ffab9be295 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3e1a2b83f3d4d88ba5084dfaffd22f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:14:46.487823 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44476505-e6a1-4b46-8cf3-c9ffab9be295 service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:46.488209 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44476505-e6a1-4b46-8cf3-c9ffab9be295 service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:46.488493 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44476505-e6a1-4b46-8cf3-c9ffab9be295 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:46.545423 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44476505-e6a1-4b46-8cf3-c9ffab9be295 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:14:46.547031 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44476505-e6a1-4b46-8cf3-c9ffab9be295 service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:14:46.556891 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1375/2719] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:14:46 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:14:49.121722 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dffa5c-59d7-4e40-9748-3ecaea112465 service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:14:49.185604 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dffa5c-59d7-4e40-9748-3ecaea112465 service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:14:49.190112 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c9ccb1-599f-4f8a-a06b-366939f37538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:14:49.191126 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c9ccb1-599f-4f8a-a06b-366939f37538 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:49.191429 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c9ccb1-599f-4f8a-a06b-366939f37538 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:49.191714 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c9ccb1-599f-4f8a-a06b-366939f37538 admin admin] PATH_INFO: `/v3/users/67bbe050db15416a919eff62d1573f63` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:49.192900 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c9ccb1-599f-4f8a-a06b-366939f37538 admin admin] RBAC: Authorizing `identity:delete_user(user_id=67bbe050db15416a919eff62d1573f63)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:14:49.194833 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c9ccb1-599f-4f8a-a06b-366939f37538 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:14:49.199501 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:14:49.246445 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-b7c9ccb1-599f-4f8a-a06b-366939f37538 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67bbe050db15416a919eff62d1573f63'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:49.254919 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-b7c9ccb1-599f-4f8a-a06b-366939f37538 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67bbe050db15416a919eff62d1573f63'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:49.266609 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-b7c9ccb1-599f-4f8a-a06b-366939f37538 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67bbe050db15416a919eff62d1573f63'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:49.267241 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-b7c9ccb1-599f-4f8a-a06b-366939f37538 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67bbe050db15416a919eff62d1573f63'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:49.268442 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-b7c9ccb1-599f-4f8a-a06b-366939f37538 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67bbe050db15416a919eff62d1573f63'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:49.280276 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1345/2720] 10.222.0.99 () {58 vars in 1223 bytes} [Thu Jul 14 16:14:49 2022] DELETE /identity/v3/users/67bbe050db15416a919eff62d1573f63 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:14:49.284999 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0fdda9-6678-4328-bbca-f41d468589df None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:49.285475 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0fdda9-6678-4328-bbca-f41d468589df None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:49.285600 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0fdda9-6678-4328-bbca-f41d468589df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:49.298055 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=100 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:14:49.304818 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-be0fdda9-6678-4328-bbca-f41d468589df None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:14:49.306041 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-be0fdda9-6678-4328-bbca-f41d468589df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:14:49.375787 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1376/2721] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 16:14:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:14:49.386090 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c9ccb1-599f-4f8a-a06b-366939f37538 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:14:49.387527 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b7c9ccb1-599f-4f8a-a06b-366939f37538 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:14:49.450603 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c9ccb1-599f-4f8a-a06b-366939f37538 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:14:49.454227 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34354623-b0a5-44ee-abe4-1f1880a016df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:14:49.454886 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34354623-b0a5-44ee-abe4-1f1880a016df service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:49.455179 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34354623-b0a5-44ee-abe4-1f1880a016df service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:49.456187 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34354623-b0a5-44ee-abe4-1f1880a016df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:49.466204 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34354623-b0a5-44ee-abe4-1f1880a016df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:14:49.467733 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34354623-b0a5-44ee-abe4-1f1880a016df service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:14:49.487860 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1346/2722] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:14:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:14:49.770070 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0fdda9-6678-4328-bbca-f41d468589df None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:14:49.771557 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-be0fdda9-6678-4328-bbca-f41d468589df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:14:49.783316 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0fdda9-6678-4328-bbca-f41d468589df None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:14:49.788696 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3400c0e-e4b1-4a9d-b51a-5b8a2df25bed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:14:49.789577 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3400c0e-e4b1-4a9d-b51a-5b8a2df25bed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:49.789888 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3400c0e-e4b1-4a9d-b51a-5b8a2df25bed admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:49.790169 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3400c0e-e4b1-4a9d-b51a-5b8a2df25bed admin admin] PATH_INFO: `/v3/projects/040d9931712242f486545152296b77a0` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:49.791365 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3400c0e-e4b1-4a9d-b51a-5b8a2df25bed admin admin] RBAC: Authorizing `identity:delete_project(project_id=040d9931712242f486545152296b77a0)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:14:49.793321 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3400c0e-e4b1-4a9d-b51a-5b8a2df25bed admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:14:49.848840 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-d3400c0e-e4b1-4a9d-b51a-5b8a2df25bed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '040d9931712242f486545152296b77a0', 'request_id': 'req-d3400c0e-e4b1-4a9d-b51a-5b8a2df25bed'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:49.860973 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-d3400c0e-e4b1-4a9d-b51a-5b8a2df25bed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '040d9931712242f486545152296b77a0', 'request_id': 'req-d3400c0e-e4b1-4a9d-b51a-5b8a2df25bed'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:49.867342 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-d3400c0e-e4b1-4a9d-b51a-5b8a2df25bed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '040d9931712242f486545152296b77a0', 'request_id': 'req-d3400c0e-e4b1-4a9d-b51a-5b8a2df25bed'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:49.881262 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-d3400c0e-e4b1-4a9d-b51a-5b8a2df25bed admin admin] The token cache is being invalidate because project 040d9931712242f486545152296b77a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 16:14:49.881558 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-d3400c0e-e4b1-4a9d-b51a-5b8a2df25bed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:49.882183 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-d3400c0e-e4b1-4a9d-b51a-5b8a2df25bed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:49.883811 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1377/2723] 10.222.0.99 () {58 vars in 1232 bytes} [Thu Jul 14 16:14:49 2022] DELETE /identity/v3/projects/040d9931712242f486545152296b77a0 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:14:49.888322 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34354623-b0a5-44ee-abe4-1f1880a016df service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:14:49.898201 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34354623-b0a5-44ee-abe4-1f1880a016df service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:14:49.957122 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4890cf-bc64-41c4-8bd4-4848c8688008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:14:49.958430 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4890cf-bc64-41c4-8bd4-4848c8688008 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:49.958718 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4890cf-bc64-41c4-8bd4-4848c8688008 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:49.958998 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4890cf-bc64-41c4-8bd4-4848c8688008 admin admin] PATH_INFO: `/v3/users/e268b9e49863439bac015fbee060fdf3` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:49.960209 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4890cf-bc64-41c4-8bd4-4848c8688008 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e268b9e49863439bac015fbee060fdf3)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:14:49.962128 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4890cf-bc64-41c4-8bd4-4848c8688008 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:14:50.007311 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:14:50.035289 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-da4890cf-bc64-41c4-8bd4-4848c8688008 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e268b9e49863439bac015fbee060fdf3'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:50.042307 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-da4890cf-bc64-41c4-8bd4-4848c8688008 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e268b9e49863439bac015fbee060fdf3'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:50.055546 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-da4890cf-bc64-41c4-8bd4-4848c8688008 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e268b9e49863439bac015fbee060fdf3'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:50.056247 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-da4890cf-bc64-41c4-8bd4-4848c8688008 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e268b9e49863439bac015fbee060fdf3'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:50.056850 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-da4890cf-bc64-41c4-8bd4-4848c8688008 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e268b9e49863439bac015fbee060fdf3'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:50.068927 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1347/2724] 10.222.0.99 () {58 vars in 1223 bytes} [Thu Jul 14 16:14:49 2022] DELETE /identity/v3/users/e268b9e49863439bac015fbee060fdf3 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:14:50.354121 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3400c0e-e4b1-4a9d-b51a-5b8a2df25bed admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:14:50.355556 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-d3400c0e-e4b1-4a9d-b51a-5b8a2df25bed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:14:50.369547 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3400c0e-e4b1-4a9d-b51a-5b8a2df25bed admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:14:50.437287 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35656637-9b93-4491-91e7-cca4f4a97a97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:14:50.438152 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35656637-9b93-4491-91e7-cca4f4a97a97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:50.438441 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35656637-9b93-4491-91e7-cca4f4a97a97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:50.438730 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35656637-9b93-4491-91e7-cca4f4a97a97 admin admin] PATH_INFO: `/v3/projects/7fdda46bd1e2466c9d5a4b1d42c45cc3` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:50.440563 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35656637-9b93-4491-91e7-cca4f4a97a97 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7fdda46bd1e2466c9d5a4b1d42c45cc3)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:14:50.443461 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35656637-9b93-4491-91e7-cca4f4a97a97 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:14:50.470215 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:14:50.500164 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-35656637-9b93-4491-91e7-cca4f4a97a97 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7fdda46bd1e2466c9d5a4b1d42c45cc3', 'request_id': 'req-35656637-9b93-4491-91e7-cca4f4a97a97'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:50.507934 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-35656637-9b93-4491-91e7-cca4f4a97a97 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7fdda46bd1e2466c9d5a4b1d42c45cc3', 'request_id': 'req-35656637-9b93-4491-91e7-cca4f4a97a97'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:50.513760 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-35656637-9b93-4491-91e7-cca4f4a97a97 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7fdda46bd1e2466c9d5a4b1d42c45cc3', 'request_id': 'req-35656637-9b93-4491-91e7-cca4f4a97a97'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:50.530516 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-35656637-9b93-4491-91e7-cca4f4a97a97 admin admin] The token cache is being invalidate because project 7fdda46bd1e2466c9d5a4b1d42c45cc3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 16:14:50.530805 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-35656637-9b93-4491-91e7-cca4f4a97a97 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:50.531450 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-35656637-9b93-4491-91e7-cca4f4a97a97 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:50.533049 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1378/2725] 10.222.0.99 () {58 vars in 1232 bytes} [Thu Jul 14 16:14:50 2022] DELETE /identity/v3/projects/7fdda46bd1e2466c9d5a4b1d42c45cc3 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:14:50.538609 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f7b98c-7eb1-4bac-ae94-fc983e6c03fc None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:50.539106 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f7b98c-7eb1-4bac-ae94-fc983e6c03fc None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:50.539214 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f7b98c-7eb1-4bac-ae94-fc983e6c03fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:50.560623 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-83f7b98c-7eb1-4bac-ae94-fc983e6c03fc None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:14:50.561927 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-83f7b98c-7eb1-4bac-ae94-fc983e6c03fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:14:50.643272 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1348/2726] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 16:14:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:14:50.651105 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42056938-b4a7-4a7d-9f41-37c49998b6b5 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:50.651405 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42056938-b4a7-4a7d-9f41-37c49998b6b5 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:50.651732 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42056938-b4a7-4a7d-9f41-37c49998b6b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:50.670657 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-42056938-b4a7-4a7d-9f41-37c49998b6b5 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:14:50.672195 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-42056938-b4a7-4a7d-9f41-37c49998b6b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:14:50.680525 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1379/2727] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 16:14:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:14:50.684757 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f7b98c-7eb1-4bac-ae94-fc983e6c03fc None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:14:50.697107 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f7b98c-7eb1-4bac-ae94-fc983e6c03fc None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:14:50.708247 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578f2ce1-9bf8-480d-ac78-b74da4f4dc18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:14:50.708970 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578f2ce1-9bf8-480d-ac78-b74da4f4dc18 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:50.709547 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578f2ce1-9bf8-480d-ac78-b74da4f4dc18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:50.710039 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578f2ce1-9bf8-480d-ac78-b74da4f4dc18 admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:50.711010 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578f2ce1-9bf8-480d-ac78-b74da4f4dc18 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 16:14:50.711327 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578f2ce1-9bf8-480d-ac78-b74da4f4dc18 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:14:50.712976 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578f2ce1-9bf8-480d-ac78-b74da4f4dc18 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:14:50.726754 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1349/2728] 10.222.0.99 () {58 vars in 1152 bytes} [Thu Jul 14 16:14:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 16:14:50.731012 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42056938-b4a7-4a7d-9f41-37c49998b6b5 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:14:50.740747 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42056938-b4a7-4a7d-9f41-37c49998b6b5 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:14:50.745767 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e18f3c-0cc9-4c6c-b7a3-26fb6680b94c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:14:50.746434 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e18f3c-0cc9-4c6c-b7a3-26fb6680b94c admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:50.746707 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e18f3c-0cc9-4c6c-b7a3-26fb6680b94c admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:50.746970 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e18f3c-0cc9-4c6c-b7a3-26fb6680b94c admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:50.747693 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e18f3c-0cc9-4c6c-b7a3-26fb6680b94c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:14:50.749589 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e18f3c-0cc9-4c6c-b7a3-26fb6680b94c admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:14:50.781065 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1380/2729] 10.222.0.99 () {60 vars in 1152 bytes} [Thu Jul 14 16:14:50 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 16:14:50.785385 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578f2ce1-9bf8-480d-ac78-b74da4f4dc18 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:14:50.795395 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578f2ce1-9bf8-480d-ac78-b74da4f4dc18 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:14:50.854433 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4467d18-6e7b-44cf-a440-a940b5cde326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:14:50.855049 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4467d18-6e7b-44cf-a440-a940b5cde326 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:50.855334 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4467d18-6e7b-44cf-a440-a940b5cde326 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:50.855633 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4467d18-6e7b-44cf-a440-a940b5cde326 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:50.856342 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4467d18-6e7b-44cf-a440-a940b5cde326 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:14:50.858283 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4467d18-6e7b-44cf-a440-a940b5cde326 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:14:50.879525 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1350/2730] 10.222.0.99 () {60 vars in 1143 bytes} [Thu Jul 14 16:14:50 2022] POST /identity/v3/users => generated 292 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 16:14:50.883676 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e18f3c-0cc9-4c6c-b7a3-26fb6680b94c admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:14:50.901306 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e18f3c-0cc9-4c6c-b7a3-26fb6680b94c admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:14:50.906106 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdba18d-e471-467a-afb7-525d849c236c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:14:50.907139 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdba18d-e471-467a-afb7-525d849c236c admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:50.907455 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdba18d-e471-467a-afb7-525d849c236c admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:50.907774 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdba18d-e471-467a-afb7-525d849c236c admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:50.908553 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdba18d-e471-467a-afb7-525d849c236c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:14:50.910169 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdba18d-e471-467a-afb7-525d849c236c admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:14:50.921273 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1381/2731] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 16:14:50 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:14:50.925999 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4467d18-6e7b-44cf-a440-a940b5cde326 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:14:50.935916 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4467d18-6e7b-44cf-a440-a940b5cde326 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:14:50.941141 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25e0dec-0d67-4e26-ba56-5a28d7bb782e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:14:50.941794 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25e0dec-0d67-4e26-ba56-5a28d7bb782e admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:50.942081 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25e0dec-0d67-4e26-ba56-5a28d7bb782e admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:50.942358 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25e0dec-0d67-4e26-ba56-5a28d7bb782e admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:50.943085 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25e0dec-0d67-4e26-ba56-5a28d7bb782e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:14:50.944713 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25e0dec-0d67-4e26-ba56-5a28d7bb782e admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:14:50.957622 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1351/2732] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 16:14:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:14:50.961868 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdba18d-e471-467a-afb7-525d849c236c admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:14:50.973104 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdba18d-e471-467a-afb7-525d849c236c admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:14:50.978132 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea789030-27b5-416e-b651-4592602bd514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:14:50.979043 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea789030-27b5-416e-b651-4592602bd514 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:50.979326 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea789030-27b5-416e-b651-4592602bd514 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:50.979664 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea789030-27b5-416e-b651-4592602bd514 admin admin] PATH_INFO: `/v3/projects/1e9f40071eb345eeb859a691e9221284/users/2a618bbe9642495093639934d3405d61/roles/b8c98fddc0fd4436b577fd1d3361bc38` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:50.996428 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea789030-27b5-416e-b651-4592602bd514 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e9f40071eb345eeb859a691e9221284, user_id=2a618bbe9642495093639934d3405d61, role_id=b8c98fddc0fd4436b577fd1d3361bc38)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:14:50.999072 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea789030-27b5-416e-b651-4592602bd514 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:14:51.012922 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1382/2733] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 16:14:50 2022] PUT /identity/v3/projects/1e9f40071eb345eeb859a691e9221284/users/2a618bbe9642495093639934d3405d61/roles/b8c98fddc0fd4436b577fd1d3361bc38 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:14:51.025138 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d872d5-2e8b-4fdd-acaf-5d56f30a89cd None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:51.025466 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d872d5-2e8b-4fdd-acaf-5d56f30a89cd None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:51.025766 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d872d5-2e8b-4fdd-acaf-5d56f30a89cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:51.045273 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:14:51.054580 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-a7d872d5-2e8b-4fdd-acaf-5d56f30a89cd None None] MFA Rules not processed for user `2a618bbe9642495093639934d3405d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:14:51.056318 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a7d872d5-2e8b-4fdd-acaf-5d56f30a89cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:14:51.131815 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1352/2734] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 16:14:51 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:14:51.144655 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea789030-27b5-416e-b651-4592602bd514 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:14:51.146387 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-ea789030-27b5-416e-b651-4592602bd514 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:14:51.205272 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea789030-27b5-416e-b651-4592602bd514 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:14:51.209408 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb657bdc-2948-4342-b398-7f48d4f2136b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3e1a2b83f3d4d88ba5084dfaffd22f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:14:51.210057 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb657bdc-2948-4342-b398-7f48d4f2136b service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:51.210341 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb657bdc-2948-4342-b398-7f48d4f2136b service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:51.210615 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb657bdc-2948-4342-b398-7f48d4f2136b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:51.219353 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb657bdc-2948-4342-b398-7f48d4f2136b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:14:51.220819 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb657bdc-2948-4342-b398-7f48d4f2136b service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:14:51.229694 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1383/2735] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:14:51 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:14:58.668450 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d872d5-2e8b-4fdd-acaf-5d56f30a89cd None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:14:58.669881 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a7d872d5-2e8b-4fdd-acaf-5d56f30a89cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:14:58.801332 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d872d5-2e8b-4fdd-acaf-5d56f30a89cd None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:14:58.806736 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efff2e67-ef14-416f-b00a-a1ccb07390b6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc157c485f794e968a60d8405c9310ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:14:58.807389 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efff2e67-ef14-416f-b00a-a1ccb07390b6 service placement] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:58.807707 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efff2e67-ef14-416f-b00a-a1ccb07390b6 service placement] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:58.807979 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efff2e67-ef14-416f-b00a-a1ccb07390b6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:58.809799 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-efff2e67-ef14-416f-b00a-a1ccb07390b6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:14:58.823007 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efff2e67-ef14-416f-b00a-a1ccb07390b6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:14:58.824681 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efff2e67-ef14-416f-b00a-a1ccb07390b6 service placement] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:14:58.837128 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1353/2736] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:14:58 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:14:59.826024 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb657bdc-2948-4342-b398-7f48d4f2136b service cinder] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:14:59.879974 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=91,keystone:DELETE=6,keystone:INSERT=3 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:14:59.893252 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb657bdc-2948-4342-b398-7f48d4f2136b service cinder] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:14:59.899994 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0398e99e-3dd2-437f-8b93-2b12a1e03bfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:14:59.900912 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0398e99e-3dd2-437f-8b93-2b12a1e03bfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:59.901202 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0398e99e-3dd2-437f-8b93-2b12a1e03bfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:59.901483 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0398e99e-3dd2-437f-8b93-2b12a1e03bfd admin admin] PATH_INFO: `/v3/users/2a618bbe9642495093639934d3405d61` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:59.902720 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0398e99e-3dd2-437f-8b93-2b12a1e03bfd admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a618bbe9642495093639934d3405d61)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:14:59.904737 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0398e99e-3dd2-437f-8b93-2b12a1e03bfd admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:14:59.955646 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-0398e99e-3dd2-437f-8b93-2b12a1e03bfd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a618bbe9642495093639934d3405d61'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:59.964571 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-0398e99e-3dd2-437f-8b93-2b12a1e03bfd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a618bbe9642495093639934d3405d61'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:59.977952 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-0398e99e-3dd2-437f-8b93-2b12a1e03bfd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a618bbe9642495093639934d3405d61'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:59.978640 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-0398e99e-3dd2-437f-8b93-2b12a1e03bfd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a618bbe9642495093639934d3405d61'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:59.979285 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-0398e99e-3dd2-437f-8b93-2b12a1e03bfd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a618bbe9642495093639934d3405d61'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:14:59.992154 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1384/2737] 10.222.0.99 () {58 vars in 1223 bytes} [Thu Jul 14 16:14:59 2022] DELETE /identity/v3/users/2a618bbe9642495093639934d3405d61 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:14:59.996999 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03c099c-60b6-4bf5-8d9f-89c48b212e4a None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:59.997497 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03c099c-60b6-4bf5-8d9f-89c48b212e4a None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:59.997620 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03c099c-60b6-4bf5-8d9f-89c48b212e4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:15:00.022336 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-d03c099c-60b6-4bf5-8d9f-89c48b212e4a None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:15:00.023577 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d03c099c-60b6-4bf5-8d9f-89c48b212e4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:15:00.086325 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1354/2738] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 16:14:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:15:00.096224 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0398e99e-3dd2-437f-8b93-2b12a1e03bfd admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:15:00.098810 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-0398e99e-3dd2-437f-8b93-2b12a1e03bfd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:15:00.157258 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0398e99e-3dd2-437f-8b93-2b12a1e03bfd admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:15:00.161884 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00529958-fa66-43ef-b52b-82dfbb205eb4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:15:00.162548 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00529958-fa66-43ef-b52b-82dfbb205eb4 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:15:00.162896 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00529958-fa66-43ef-b52b-82dfbb205eb4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:15:00.163207 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00529958-fa66-43ef-b52b-82dfbb205eb4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:15:00.174628 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00529958-fa66-43ef-b52b-82dfbb205eb4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:15:00.176182 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00529958-fa66-43ef-b52b-82dfbb205eb4 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:15:00.186434 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1385/2739] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:15:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:15:00.473323 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03c099c-60b6-4bf5-8d9f-89c48b212e4a None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:15:00.474977 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d03c099c-60b6-4bf5-8d9f-89c48b212e4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:15:00.490206 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03c099c-60b6-4bf5-8d9f-89c48b212e4a None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:15:00.495057 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbcfaf1-43ba-459c-aba9-cf9be8d3ef7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:15:00.495988 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbcfaf1-43ba-459c-aba9-cf9be8d3ef7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:15:00.496274 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbcfaf1-43ba-459c-aba9-cf9be8d3ef7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:15:00.497443 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbcfaf1-43ba-459c-aba9-cf9be8d3ef7b admin admin] PATH_INFO: `/v3/projects/1e9f40071eb345eeb859a691e9221284` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:15:00.499180 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbcfaf1-43ba-459c-aba9-cf9be8d3ef7b admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e9f40071eb345eeb859a691e9221284)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:15:00.501084 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbcfaf1-43ba-459c-aba9-cf9be8d3ef7b admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:15:00.509217 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:15:00.552410 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-0dbcfaf1-43ba-459c-aba9-cf9be8d3ef7b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e9f40071eb345eeb859a691e9221284', 'request_id': 'req-0dbcfaf1-43ba-459c-aba9-cf9be8d3ef7b'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:15:00.558575 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-0dbcfaf1-43ba-459c-aba9-cf9be8d3ef7b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e9f40071eb345eeb859a691e9221284', 'request_id': 'req-0dbcfaf1-43ba-459c-aba9-cf9be8d3ef7b'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:15:00.564344 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-0dbcfaf1-43ba-459c-aba9-cf9be8d3ef7b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e9f40071eb345eeb859a691e9221284', 'request_id': 'req-0dbcfaf1-43ba-459c-aba9-cf9be8d3ef7b'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:15:00.576838 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-0dbcfaf1-43ba-459c-aba9-cf9be8d3ef7b admin admin] The token cache is being invalidate because project 1e9f40071eb345eeb859a691e9221284 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 16:15:00.577138 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-0dbcfaf1-43ba-459c-aba9-cf9be8d3ef7b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:15:00.577765 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-0dbcfaf1-43ba-459c-aba9-cf9be8d3ef7b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:15:00.579317 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1355/2740] 10.222.0.99 () {58 vars in 1232 bytes} [Thu Jul 14 16:15:00 2022] DELETE /identity/v3/projects/1e9f40071eb345eeb859a691e9221284 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:15:00.584664 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aec4cdb-f8e4-4ea1-a642-81054fea14e1 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:15:00.585010 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aec4cdb-f8e4-4ea1-a642-81054fea14e1 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:15:00.585406 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aec4cdb-f8e4-4ea1-a642-81054fea14e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:15:00.603575 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-8aec4cdb-f8e4-4ea1-a642-81054fea14e1 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:15:00.604736 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-8aec4cdb-f8e4-4ea1-a642-81054fea14e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:15:00.667717 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1386/2741] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 16:15:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:15:00.676705 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf801d5-6071-498e-811f-546f614c6afe None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:15:00.677002 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf801d5-6071-498e-811f-546f614c6afe None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:15:00.677287 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf801d5-6071-498e-811f-546f614c6afe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:15:00.695938 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-fbf801d5-6071-498e-811f-546f614c6afe None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:15:00.697157 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-fbf801d5-6071-498e-811f-546f614c6afe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:15:00.705007 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1356/2742] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 16:15:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:15:00.709688 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aec4cdb-f8e4-4ea1-a642-81054fea14e1 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:15:00.719606 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aec4cdb-f8e4-4ea1-a642-81054fea14e1 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:15:00.728521 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c87ce05-4c43-4d5a-ab76-dbf520602bc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:15:00.729191 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c87ce05-4c43-4d5a-ab76-dbf520602bc0 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:15:00.729511 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c87ce05-4c43-4d5a-ab76-dbf520602bc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:15:00.729788 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c87ce05-4c43-4d5a-ab76-dbf520602bc0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:15:00.730545 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c87ce05-4c43-4d5a-ab76-dbf520602bc0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 16:15:00.730855 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c87ce05-4c43-4d5a-ab76-dbf520602bc0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:15:00.732488 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c87ce05-4c43-4d5a-ab76-dbf520602bc0 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:15:00.742002 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:15:00.745815 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1387/2743] 10.222.0.99 () {58 vars in 1152 bytes} [Thu Jul 14 16:15:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 16:15:00.750214 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf801d5-6071-498e-811f-546f614c6afe None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:15:00.759893 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf801d5-6071-498e-811f-546f614c6afe None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:15:00.764641 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9cd97b-6838-4f7d-b10d-98588399e28f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:15:00.765301 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9cd97b-6838-4f7d-b10d-98588399e28f admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:15:00.765591 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9cd97b-6838-4f7d-b10d-98588399e28f admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:15:00.765874 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9cd97b-6838-4f7d-b10d-98588399e28f admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:15:00.766568 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9cd97b-6838-4f7d-b10d-98588399e28f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:15:00.768556 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9cd97b-6838-4f7d-b10d-98588399e28f admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:15:00.794579 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1357/2744] 10.222.0.99 () {60 vars in 1152 bytes} [Thu Jul 14 16:15:00 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 16:15:00.798969 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c87ce05-4c43-4d5a-ab76-dbf520602bc0 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:15:00.809491 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c87ce05-4c43-4d5a-ab76-dbf520602bc0 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:15:00.867292 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e285aa-bfba-4916-aa79-5036a47b8a1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:15:00.868014 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e285aa-bfba-4916-aa79-5036a47b8a1e admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:15:00.868183 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e285aa-bfba-4916-aa79-5036a47b8a1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:15:00.868464 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e285aa-bfba-4916-aa79-5036a47b8a1e admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:15:00.869160 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e285aa-bfba-4916-aa79-5036a47b8a1e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:15:00.871078 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e285aa-bfba-4916-aa79-5036a47b8a1e admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:15:00.890497 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1388/2745] 10.222.0.99 () {60 vars in 1143 bytes} [Thu Jul 14 16:15:00 2022] POST /identity/v3/users => generated 293 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 16:15:00.895309 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9cd97b-6838-4f7d-b10d-98588399e28f admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:15:00.905959 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9cd97b-6838-4f7d-b10d-98588399e28f admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:15:00.911739 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5896f1a7-4246-4a61-bc8d-47c0de012d3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:15:00.912462 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5896f1a7-4246-4a61-bc8d-47c0de012d3f admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:15:00.912747 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5896f1a7-4246-4a61-bc8d-47c0de012d3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:15:00.913028 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5896f1a7-4246-4a61-bc8d-47c0de012d3f admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:15:00.913792 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5896f1a7-4246-4a61-bc8d-47c0de012d3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:15:00.915393 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5896f1a7-4246-4a61-bc8d-47c0de012d3f admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:15:00.927089 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1358/2746] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 16:15:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:15:00.931394 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e285aa-bfba-4916-aa79-5036a47b8a1e admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:15:00.943007 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e285aa-bfba-4916-aa79-5036a47b8a1e admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:15:00.946996 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab7dc2c-f638-4b99-9ee7-14e0cc5af3b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:15:00.947704 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab7dc2c-f638-4b99-9ee7-14e0cc5af3b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:15:00.947999 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab7dc2c-f638-4b99-9ee7-14e0cc5af3b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:15:00.948285 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab7dc2c-f638-4b99-9ee7-14e0cc5af3b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:15:00.949024 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab7dc2c-f638-4b99-9ee7-14e0cc5af3b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:15:00.950633 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab7dc2c-f638-4b99-9ee7-14e0cc5af3b2 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:15:00.961376 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1389/2747] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 16:15:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:15:00.965572 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5896f1a7-4246-4a61-bc8d-47c0de012d3f admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:15:00.977466 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5896f1a7-4246-4a61-bc8d-47c0de012d3f admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:15:00.982287 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bc6710-e9b7-4538-9555-97968328392a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:15:00.983183 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bc6710-e9b7-4538-9555-97968328392a admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:15:00.983467 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bc6710-e9b7-4538-9555-97968328392a admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:15:00.983784 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bc6710-e9b7-4538-9555-97968328392a admin admin] PATH_INFO: `/v3/projects/704af3de4d124e4c9dcdbba53dd79f0e/users/9e516a2fa95d4661accf1d6447666762/roles/b8c98fddc0fd4436b577fd1d3361bc38` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:15:01.001866 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bc6710-e9b7-4538-9555-97968328392a admin admin] RBAC: Authorizing `identity:create_grant(project_id=704af3de4d124e4c9dcdbba53dd79f0e, user_id=9e516a2fa95d4661accf1d6447666762, role_id=b8c98fddc0fd4436b577fd1d3361bc38)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:15:01.005100 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bc6710-e9b7-4538-9555-97968328392a admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:15:01.018973 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1359/2748] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 16:15:00 2022] PUT /identity/v3/projects/704af3de4d124e4c9dcdbba53dd79f0e/users/9e516a2fa95d4661accf1d6447666762/roles/b8c98fddc0fd4436b577fd1d3361bc38 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:15:01.029834 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0aa494-02e7-4c8e-b4b4-2d1bb7da8b83 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:15:01.030134 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0aa494-02e7-4c8e-b4b4-2d1bb7da8b83 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:15:01.030422 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0aa494-02e7-4c8e-b4b4-2d1bb7da8b83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:15:01.059285 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-5a0aa494-02e7-4c8e-b4b4-2d1bb7da8b83 None None] MFA Rules not processed for user `9e516a2fa95d4661accf1d6447666762`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:15:01.060581 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-5a0aa494-02e7-4c8e-b4b4-2d1bb7da8b83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:15:01.129015 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1390/2749] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 16:15:01 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:15:01.140229 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bc6710-e9b7-4538-9555-97968328392a admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:15:01.142006 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a9bc6710-e9b7-4538-9555-97968328392a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:15:01.202194 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bc6710-e9b7-4538-9555-97968328392a admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:15:01.205950 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de09f26-de5d-4841-a967-374bdc8c0aa2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3e1a2b83f3d4d88ba5084dfaffd22f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:15:01.207369 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de09f26-de5d-4841-a967-374bdc8c0aa2 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:15:01.207519 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de09f26-de5d-4841-a967-374bdc8c0aa2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:15:01.207755 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de09f26-de5d-4841-a967-374bdc8c0aa2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:15:01.216434 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de09f26-de5d-4841-a967-374bdc8c0aa2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:15:01.217875 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de09f26-de5d-4841-a967-374bdc8c0aa2 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:15:01.226860 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1360/2750] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:15:01 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:15:03.782939 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0aa494-02e7-4c8e-b4b4-2d1bb7da8b83 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:15:03.846281 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0aa494-02e7-4c8e-b4b4-2d1bb7da8b83 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:15:03.851063 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8a257e-84bf-4995-81a3-06d493f42e28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:15:03.852055 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8a257e-84bf-4995-81a3-06d493f42e28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:15:03.852345 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8a257e-84bf-4995-81a3-06d493f42e28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:15:03.852626 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8a257e-84bf-4995-81a3-06d493f42e28 admin admin] PATH_INFO: `/v3/users/9e516a2fa95d4661accf1d6447666762` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:15:03.853821 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8a257e-84bf-4995-81a3-06d493f42e28 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e516a2fa95d4661accf1d6447666762)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:15:03.855804 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8a257e-84bf-4995-81a3-06d493f42e28 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:15:03.866920 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:15:03.905070 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-ba8a257e-84bf-4995-81a3-06d493f42e28 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e516a2fa95d4661accf1d6447666762'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:15:03.912858 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-ba8a257e-84bf-4995-81a3-06d493f42e28 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e516a2fa95d4661accf1d6447666762'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:15:03.926874 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-ba8a257e-84bf-4995-81a3-06d493f42e28 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e516a2fa95d4661accf1d6447666762'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:15:03.927626 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-ba8a257e-84bf-4995-81a3-06d493f42e28 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e516a2fa95d4661accf1d6447666762'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:15:03.928343 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-ba8a257e-84bf-4995-81a3-06d493f42e28 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e516a2fa95d4661accf1d6447666762'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:15:03.940543 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1391/2751] 10.222.0.99 () {58 vars in 1223 bytes} [Thu Jul 14 16:15:03 2022] DELETE /identity/v3/users/9e516a2fa95d4661accf1d6447666762 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:15:03.946010 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31245a88-e780-4569-a113-1c34f92d9d6a None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:15:03.946523 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31245a88-e780-4569-a113-1c34f92d9d6a None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:15:03.946645 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31245a88-e780-4569-a113-1c34f92d9d6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:15:03.963692 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-31245a88-e780-4569-a113-1c34f92d9d6a None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:15:03.964906 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-31245a88-e780-4569-a113-1c34f92d9d6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:15:03.997075 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:15:04.030336 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1361/2752] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 16:15:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:15:04.040546 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8a257e-84bf-4995-81a3-06d493f42e28 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:15:04.042007 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-ba8a257e-84bf-4995-81a3-06d493f42e28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:15:04.101401 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8a257e-84bf-4995-81a3-06d493f42e28 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:15:04.105048 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58df13b-aef5-457b-9707-ff21e00ac956 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:15:04.105696 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58df13b-aef5-457b-9707-ff21e00ac956 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:15:04.105980 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58df13b-aef5-457b-9707-ff21e00ac956 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:15:04.106257 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58df13b-aef5-457b-9707-ff21e00ac956 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:15:04.115988 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58df13b-aef5-457b-9707-ff21e00ac956 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:15:04.117424 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58df13b-aef5-457b-9707-ff21e00ac956 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:15:04.126990 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1392/2753] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:15:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:15:04.420293 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31245a88-e780-4569-a113-1c34f92d9d6a None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:15:04.422303 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-31245a88-e780-4569-a113-1c34f92d9d6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:15:04.437475 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31245a88-e780-4569-a113-1c34f92d9d6a None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:15:04.442774 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5891c4ec-6f4e-43f9-9691-51d6d8a254ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:15:04.443643 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5891c4ec-6f4e-43f9-9691-51d6d8a254ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:15:04.443922 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5891c4ec-6f4e-43f9-9691-51d6d8a254ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:15:04.444595 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5891c4ec-6f4e-43f9-9691-51d6d8a254ce admin admin] PATH_INFO: `/v3/projects/704af3de4d124e4c9dcdbba53dd79f0e` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:15:04.445595 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5891c4ec-6f4e-43f9-9691-51d6d8a254ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=704af3de4d124e4c9dcdbba53dd79f0e)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:15:04.447524 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5891c4ec-6f4e-43f9-9691-51d6d8a254ce admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:15:04.515812 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5891c4ec-6f4e-43f9-9691-51d6d8a254ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '704af3de4d124e4c9dcdbba53dd79f0e', 'request_id': 'req-5891c4ec-6f4e-43f9-9691-51d6d8a254ce'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:15:04.530420 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5891c4ec-6f4e-43f9-9691-51d6d8a254ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '704af3de4d124e4c9dcdbba53dd79f0e', 'request_id': 'req-5891c4ec-6f4e-43f9-9691-51d6d8a254ce'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:15:04.540819 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5891c4ec-6f4e-43f9-9691-51d6d8a254ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '704af3de4d124e4c9dcdbba53dd79f0e', 'request_id': 'req-5891c4ec-6f4e-43f9-9691-51d6d8a254ce'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:15:04.555922 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5891c4ec-6f4e-43f9-9691-51d6d8a254ce admin admin] The token cache is being invalidate because project 704af3de4d124e4c9dcdbba53dd79f0e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 16:15:04.556193 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5891c4ec-6f4e-43f9-9691-51d6d8a254ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:15:04.557015 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5891c4ec-6f4e-43f9-9691-51d6d8a254ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:15:04.558616 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1362/2754] 10.222.0.99 () {58 vars in 1232 bytes} [Thu Jul 14 16:15:04 2022] DELETE /identity/v3/projects/704af3de4d124e4c9dcdbba53dd79f0e => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 16:15:04.563671 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d966033-9de4-429a-8b08-bdc8dbcb992b None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:15:04.564174 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d966033-9de4-429a-8b08-bdc8dbcb992b None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:15:04.564468 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d966033-9de4-429a-8b08-bdc8dbcb992b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:15:04.584427 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-6d966033-9de4-429a-8b08-bdc8dbcb992b None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:15:04.585512 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-6d966033-9de4-429a-8b08-bdc8dbcb992b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:15:04.661870 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1393/2755] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 16:15:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:15:04.726940 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8926903a-b271-4f94-acfb-d61fcd7f68d6 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:15:04.727230 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8926903a-b271-4f94-acfb-d61fcd7f68d6 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:15:04.727506 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8926903a-b271-4f94-acfb-d61fcd7f68d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:15:04.747138 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-8926903a-b271-4f94-acfb-d61fcd7f68d6 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:15:04.748418 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-8926903a-b271-4f94-acfb-d61fcd7f68d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:15:04.756933 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1363/2756] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 16:15:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:15:04.761685 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d966033-9de4-429a-8b08-bdc8dbcb992b None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:15:04.772642 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d966033-9de4-429a-8b08-bdc8dbcb992b None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:15:04.777397 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79ea451-9a3e-4cc1-99cc-c99aebbe0954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:15:04.778029 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79ea451-9a3e-4cc1-99cc-c99aebbe0954 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:15:04.778988 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79ea451-9a3e-4cc1-99cc-c99aebbe0954 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:15:04.778988 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79ea451-9a3e-4cc1-99cc-c99aebbe0954 admin admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:15:04.779320 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79ea451-9a3e-4cc1-99cc-c99aebbe0954 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 16:15:04.779664 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79ea451-9a3e-4cc1-99cc-c99aebbe0954 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:15:04.781281 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79ea451-9a3e-4cc1-99cc-c99aebbe0954 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:15:04.795739 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1394/2757] 10.222.0.99 () {58 vars in 1152 bytes} [Thu Jul 14 16:15:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 16:15:04.800856 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8926903a-b271-4f94-acfb-d61fcd7f68d6 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:15:04.810509 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8926903a-b271-4f94-acfb-d61fcd7f68d6 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:15:04.815301 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd846e4-324a-468f-a0af-de51d52a1d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:15:04.816201 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd846e4-324a-468f-a0af-de51d52a1d16 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:15:04.816497 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd846e4-324a-468f-a0af-de51d52a1d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:15:04.816773 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd846e4-324a-468f-a0af-de51d52a1d16 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:15:04.817485 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd846e4-324a-468f-a0af-de51d52a1d16 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:15:04.819501 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd846e4-324a-468f-a0af-de51d52a1d16 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:15:04.850782 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1364/2758] 10.222.0.99 () {60 vars in 1152 bytes} [Thu Jul 14 16:15:04 2022] POST /identity/v3/projects => generated 371 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 16:15:04.854882 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79ea451-9a3e-4cc1-99cc-c99aebbe0954 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:15:04.865239 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79ea451-9a3e-4cc1-99cc-c99aebbe0954 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:15:04.878145 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:15:04.929473 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5539c3f9-04b4-4d6a-a93a-769b51de983e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:15:04.930161 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5539c3f9-04b4-4d6a-a93a-769b51de983e admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:15:04.930449 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5539c3f9-04b4-4d6a-a93a-769b51de983e admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:15:04.930731 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5539c3f9-04b4-4d6a-a93a-769b51de983e admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:15:04.931433 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5539c3f9-04b4-4d6a-a93a-769b51de983e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:15:04.933427 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5539c3f9-04b4-4d6a-a93a-769b51de983e admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:15:04.954263 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1395/2759] 10.222.0.99 () {60 vars in 1143 bytes} [Thu Jul 14 16:15:04 2022] POST /identity/v3/users => generated 286 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 16:15:04.959557 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd846e4-324a-468f-a0af-de51d52a1d16 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:15:04.970004 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd846e4-324a-468f-a0af-de51d52a1d16 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:15:04.975207 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a23f40b-757e-4490-9dd1-0e489abbc634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:15:04.975910 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a23f40b-757e-4490-9dd1-0e489abbc634 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:15:04.976202 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a23f40b-757e-4490-9dd1-0e489abbc634 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:15:04.976488 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a23f40b-757e-4490-9dd1-0e489abbc634 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:15:04.977232 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a23f40b-757e-4490-9dd1-0e489abbc634 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:15:04.978837 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a23f40b-757e-4490-9dd1-0e489abbc634 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:15:04.990380 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1365/2760] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 16:15:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 16:15:04.994225 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5539c3f9-04b4-4d6a-a93a-769b51de983e admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:15:05.005362 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5539c3f9-04b4-4d6a-a93a-769b51de983e admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:15:05.010269 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b03736-25da-4961-8c60-f716bc66df7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:15:05.010958 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b03736-25da-4961-8c60-f716bc66df7c admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:15:05.011247 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b03736-25da-4961-8c60-f716bc66df7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:15:05.011526 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b03736-25da-4961-8c60-f716bc66df7c admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:15:05.012287 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b03736-25da-4961-8c60-f716bc66df7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:15:05.013890 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b03736-25da-4961-8c60-f716bc66df7c admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:15:05.025110 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1396/2761] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 16:15:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:15:05.029463 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a23f40b-757e-4490-9dd1-0e489abbc634 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:15:05.041080 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a23f40b-757e-4490-9dd1-0e489abbc634 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:15:05.046133 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe68421-96a4-457d-a5c4-9d18e76bc727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:15:05.047657 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe68421-96a4-457d-a5c4-9d18e76bc727 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:15:05.047942 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe68421-96a4-457d-a5c4-9d18e76bc727 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:15:05.048219 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe68421-96a4-457d-a5c4-9d18e76bc727 admin admin] PATH_INFO: `/v3/projects/3f593e8ef35e4e608fd1070db5375c3a/users/48bf6ac7a7f84cc490745e055c71253c/roles/b8c98fddc0fd4436b577fd1d3361bc38` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:15:05.063482 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe68421-96a4-457d-a5c4-9d18e76bc727 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f593e8ef35e4e608fd1070db5375c3a, user_id=48bf6ac7a7f84cc490745e055c71253c, role_id=b8c98fddc0fd4436b577fd1d3361bc38)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:15:05.067087 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe68421-96a4-457d-a5c4-9d18e76bc727 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:15:05.081144 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1366/2762] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 16:15:05 2022] PUT /identity/v3/projects/3f593e8ef35e4e608fd1070db5375c3a/users/48bf6ac7a7f84cc490745e055c71253c/roles/b8c98fddc0fd4436b577fd1d3361bc38 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 16:15:05.093621 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f90a4d-3c5c-4f01-b6a8-396ff4e37de9 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:15:05.093847 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f90a4d-3c5c-4f01-b6a8-396ff4e37de9 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:15:05.094134 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f90a4d-3c5c-4f01-b6a8-396ff4e37de9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:15:05.123865 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-f7f90a4d-3c5c-4f01-b6a8-396ff4e37de9 None None] MFA Rules not processed for user `48bf6ac7a7f84cc490745e055c71253c`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:15:05.125088 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-f7f90a4d-3c5c-4f01-b6a8-396ff4e37de9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:15:05.193482 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1397/2763] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 16:15:05 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 16:15:05.204329 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe68421-96a4-457d-a5c4-9d18e76bc727 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:15:05.206105 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0fe68421-96a4-457d-a5c4-9d18e76bc727 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:15:05.267631 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe68421-96a4-457d-a5c4-9d18e76bc727 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:15:05.272969 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a7f6e3-2f52-4797-b0b8-c31dc4ce5cc5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3e1a2b83f3d4d88ba5084dfaffd22f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:15:05.273202 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a7f6e3-2f52-4797-b0b8-c31dc4ce5cc5 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:15:05.273489 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a7f6e3-2f52-4797-b0b8-c31dc4ce5cc5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:15:05.273769 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a7f6e3-2f52-4797-b0b8-c31dc4ce5cc5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:15:05.280500 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:15:05.283092 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a7f6e3-2f52-4797-b0b8-c31dc4ce5cc5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:15:05.285208 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a7f6e3-2f52-4797-b0b8-c31dc4ce5cc5 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:15:05.297830 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1367/2764] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:15:05 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:15:05.323549 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f90a4d-3c5c-4f01-b6a8-396ff4e37de9 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:15:05.401419 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f90a4d-3c5c-4f01-b6a8-396ff4e37de9 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:15:05.406104 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d01e854-39bd-4d63-a7fe-8c534362bc58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:15:05.406982 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d01e854-39bd-4d63-a7fe-8c534362bc58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:15:05.407270 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d01e854-39bd-4d63-a7fe-8c534362bc58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:15:05.407552 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d01e854-39bd-4d63-a7fe-8c534362bc58 admin admin] PATH_INFO: `/v3/users/48bf6ac7a7f84cc490745e055c71253c` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:15:05.408879 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d01e854-39bd-4d63-a7fe-8c534362bc58 admin admin] RBAC: Authorizing `identity:delete_user(user_id=48bf6ac7a7f84cc490745e055c71253c)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:15:05.411269 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d01e854-39bd-4d63-a7fe-8c534362bc58 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:15:05.442636 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:15:05.466114 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-5d01e854-39bd-4d63-a7fe-8c534362bc58 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48bf6ac7a7f84cc490745e055c71253c'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:15:05.475324 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-5d01e854-39bd-4d63-a7fe-8c534362bc58 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48bf6ac7a7f84cc490745e055c71253c'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:15:05.493718 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-5d01e854-39bd-4d63-a7fe-8c534362bc58 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48bf6ac7a7f84cc490745e055c71253c'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:15:05.494443 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-5d01e854-39bd-4d63-a7fe-8c534362bc58 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48bf6ac7a7f84cc490745e055c71253c'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:15:05.495054 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-5d01e854-39bd-4d63-a7fe-8c534362bc58 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48bf6ac7a7f84cc490745e055c71253c'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:15:05.509180 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1398/2765] 10.222.0.99 () {58 vars in 1223 bytes} [Thu Jul 14 16:15:05 2022] DELETE /identity/v3/users/48bf6ac7a7f84cc490745e055c71253c => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:15:05.513743 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828e73b9-88ff-4a56-99ff-6be9b9ba2c57 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:15:05.514042 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828e73b9-88ff-4a56-99ff-6be9b9ba2c57 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:15:05.514331 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828e73b9-88ff-4a56-99ff-6be9b9ba2c57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:15:05.537078 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-828e73b9-88ff-4a56-99ff-6be9b9ba2c57 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:15:05.538319 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-828e73b9-88ff-4a56-99ff-6be9b9ba2c57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:15:05.613522 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1368/2766] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 16:15:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:15:05.623544 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d01e854-39bd-4d63-a7fe-8c534362bc58 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:15:05.625051 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-5d01e854-39bd-4d63-a7fe-8c534362bc58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:15:05.712607 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d01e854-39bd-4d63-a7fe-8c534362bc58 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:15:05.718837 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb556a73-91b1-49d3-90a8-38e7e6e3304e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:15:05.720035 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb556a73-91b1-49d3-90a8-38e7e6e3304e service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:15:05.720570 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb556a73-91b1-49d3-90a8-38e7e6e3304e service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:15:05.721087 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb556a73-91b1-49d3-90a8-38e7e6e3304e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:15:05.738285 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb556a73-91b1-49d3-90a8-38e7e6e3304e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:15:05.740867 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb556a73-91b1-49d3-90a8-38e7e6e3304e service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:15:05.753084 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1399/2767] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:15:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:15:06.070935 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828e73b9-88ff-4a56-99ff-6be9b9ba2c57 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:15:06.072890 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-828e73b9-88ff-4a56-99ff-6be9b9ba2c57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:15:06.088454 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828e73b9-88ff-4a56-99ff-6be9b9ba2c57 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:15:06.093312 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b759cf-94f2-43fe-81db-e10e838f28b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:15:06.094198 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b759cf-94f2-43fe-81db-e10e838f28b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:15:06.094941 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b759cf-94f2-43fe-81db-e10e838f28b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:15:06.094941 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b759cf-94f2-43fe-81db-e10e838f28b6 admin admin] PATH_INFO: `/v3/projects/3f593e8ef35e4e608fd1070db5375c3a` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:15:06.096102 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b759cf-94f2-43fe-81db-e10e838f28b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f593e8ef35e4e608fd1070db5375c3a)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:15:06.098339 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b759cf-94f2-43fe-81db-e10e838f28b6 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:15:06.164231 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-56b759cf-94f2-43fe-81db-e10e838f28b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f593e8ef35e4e608fd1070db5375c3a', 'request_id': 'req-56b759cf-94f2-43fe-81db-e10e838f28b6'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:15:06.173185 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-56b759cf-94f2-43fe-81db-e10e838f28b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f593e8ef35e4e608fd1070db5375c3a', 'request_id': 'req-56b759cf-94f2-43fe-81db-e10e838f28b6'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:15:06.183201 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-56b759cf-94f2-43fe-81db-e10e838f28b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f593e8ef35e4e608fd1070db5375c3a', 'request_id': 'req-56b759cf-94f2-43fe-81db-e10e838f28b6'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:15:06.198451 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-56b759cf-94f2-43fe-81db-e10e838f28b6 admin admin] The token cache is being invalidate because project 3f593e8ef35e4e608fd1070db5375c3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 16:15:06.198738 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-56b759cf-94f2-43fe-81db-e10e838f28b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:15:06.199382 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-56b759cf-94f2-43fe-81db-e10e838f28b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:15:06.200998 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1369/2768] 10.222.0.99 () {58 vars in 1232 bytes} [Thu Jul 14 16:15:06 2022] DELETE /identity/v3/projects/3f593e8ef35e4e608fd1070db5375c3a => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 16:15:06.206053 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58ec1fe-f2ed-4285-8a70-2793cdb3623c None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:15:06.207154 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58ec1fe-f2ed-4285-8a70-2793cdb3623c None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:15:06.207154 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58ec1fe-f2ed-4285-8a70-2793cdb3623c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:15:06.225915 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-c58ec1fe-f2ed-4285-8a70-2793cdb3623c None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:15:06.227215 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-c58ec1fe-f2ed-4285-8a70-2793cdb3623c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:15:06.294183 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1400/2769] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 16:15:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:15:06.303248 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df9f280-8704-405a-89b7-5742bb3f64ea None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:15:06.303532 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df9f280-8704-405a-89b7-5742bb3f64ea None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:15:06.303846 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df9f280-8704-405a-89b7-5742bb3f64ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:15:06.323778 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-7df9f280-8704-405a-89b7-5742bb3f64ea None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:15:06.325019 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-7df9f280-8704-405a-89b7-5742bb3f64ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:15:06.333599 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1370/2770] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 16:15:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:15:06.338824 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58ec1fe-f2ed-4285-8a70-2793cdb3623c None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:15:06.359454 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58ec1fe-f2ed-4285-8a70-2793cdb3623c None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:15:06.365404 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3151d1-6270-4735-9307-4f69ea95d15c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:15:06.366082 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3151d1-6270-4735-9307-4f69ea95d15c admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:15:06.366370 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3151d1-6270-4735-9307-4f69ea95d15c admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:15:06.366706 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3151d1-6270-4735-9307-4f69ea95d15c admin admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:15:06.368085 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3151d1-6270-4735-9307-4f69ea95d15c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 16:15:06.368630 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3151d1-6270-4735-9307-4f69ea95d15c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:15:06.371614 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3151d1-6270-4735-9307-4f69ea95d15c admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:15:06.388094 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1401/2771] 10.222.0.99 () {58 vars in 1152 bytes} [Thu Jul 14 16:15:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 16:15:06.392711 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df9f280-8704-405a-89b7-5742bb3f64ea None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:15:06.403070 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df9f280-8704-405a-89b7-5742bb3f64ea None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:15:06.407821 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9ea2e9-0cb8-4f51-b88f-e8fb6c339ff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:15:06.408486 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9ea2e9-0cb8-4f51-b88f-e8fb6c339ff7 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:15:06.408774 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9ea2e9-0cb8-4f51-b88f-e8fb6c339ff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:15:06.409052 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9ea2e9-0cb8-4f51-b88f-e8fb6c339ff7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:15:06.410632 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9ea2e9-0cb8-4f51-b88f-e8fb6c339ff7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:15:06.412667 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9ea2e9-0cb8-4f51-b88f-e8fb6c339ff7 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:15:06.454609 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1371/2772] 10.222.0.99 () {60 vars in 1152 bytes} [Thu Jul 14 16:15:06 2022] POST /identity/v3/projects => generated 391 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 16:15:06.458875 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3151d1-6270-4735-9307-4f69ea95d15c admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:15:06.469635 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3151d1-6270-4735-9307-4f69ea95d15c admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:15:06.504114 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:15:06.529229 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0cc172-673c-4a80-a2cd-48617e23102a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:15:06.529905 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0cc172-673c-4a80-a2cd-48617e23102a admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:15:06.530190 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0cc172-673c-4a80-a2cd-48617e23102a admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:15:06.530467 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0cc172-673c-4a80-a2cd-48617e23102a admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:15:06.531160 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0cc172-673c-4a80-a2cd-48617e23102a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:15:06.533094 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0cc172-673c-4a80-a2cd-48617e23102a admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:15:06.555528 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1402/2773] 10.222.0.99 () {60 vars in 1143 bytes} [Thu Jul 14 16:15:06 2022] POST /identity/v3/users => generated 296 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 16:15:06.559588 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9ea2e9-0cb8-4f51-b88f-e8fb6c339ff7 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:15:06.570526 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9ea2e9-0cb8-4f51-b88f-e8fb6c339ff7 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:15:06.575349 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3463917-bc6c-409a-a439-204d84b02821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:15:06.576068 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3463917-bc6c-409a-a439-204d84b02821 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:15:06.576356 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3463917-bc6c-409a-a439-204d84b02821 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:15:06.576640 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3463917-bc6c-409a-a439-204d84b02821 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:15:06.577380 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3463917-bc6c-409a-a439-204d84b02821 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:15:06.578978 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3463917-bc6c-409a-a439-204d84b02821 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:15:06.591293 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1372/2774] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 16:15:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:15:06.595115 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0cc172-673c-4a80-a2cd-48617e23102a admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:15:06.607624 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0cc172-673c-4a80-a2cd-48617e23102a admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:15:06.612415 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c901559-d15a-4edb-847f-5b97f2e79768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:15:06.613820 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c901559-d15a-4edb-847f-5b97f2e79768 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:15:06.614099 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c901559-d15a-4edb-847f-5b97f2e79768 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:15:06.614376 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c901559-d15a-4edb-847f-5b97f2e79768 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:15:06.615112 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c901559-d15a-4edb-847f-5b97f2e79768 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:15:06.616761 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c901559-d15a-4edb-847f-5b97f2e79768 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:15:06.628022 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1403/2775] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 16:15:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:15:06.632489 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3463917-bc6c-409a-a439-204d84b02821 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:15:06.643155 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3463917-bc6c-409a-a439-204d84b02821 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:15:06.648068 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7151274-3519-4ff5-a9b0-2de6e4e405c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:15:06.648971 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7151274-3519-4ff5-a9b0-2de6e4e405c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:15:06.649263 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7151274-3519-4ff5-a9b0-2de6e4e405c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:15:06.649553 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7151274-3519-4ff5-a9b0-2de6e4e405c0 admin admin] PATH_INFO: `/v3/projects/f8226bb5ee7e40e8af9a8029c5128b23/users/faeec826cf964066bda85d1e20ba0da9/roles/b8c98fddc0fd4436b577fd1d3361bc38` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:15:06.678192 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7151274-3519-4ff5-a9b0-2de6e4e405c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8226bb5ee7e40e8af9a8029c5128b23, user_id=faeec826cf964066bda85d1e20ba0da9, role_id=b8c98fddc0fd4436b577fd1d3361bc38)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:15:06.681161 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7151274-3519-4ff5-a9b0-2de6e4e405c0 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:15:06.696363 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1373/2776] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 16:15:06 2022] PUT /identity/v3/projects/f8226bb5ee7e40e8af9a8029c5128b23/users/faeec826cf964066bda85d1e20ba0da9/roles/b8c98fddc0fd4436b577fd1d3361bc38 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 16:15:06.708442 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079949cb-5dfc-4737-877a-b81d48830459 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:15:06.708930 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079949cb-5dfc-4737-877a-b81d48830459 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:15:06.709283 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079949cb-5dfc-4737-877a-b81d48830459 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:15:06.738493 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-079949cb-5dfc-4737-877a-b81d48830459 None None] MFA Rules not processed for user `faeec826cf964066bda85d1e20ba0da9`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:15:06.739783 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-079949cb-5dfc-4737-877a-b81d48830459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:15:06.809524 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1404/2777] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 16:15:06 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:15:06.820434 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7151274-3519-4ff5-a9b0-2de6e4e405c0 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:15:06.822374 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a7151274-3519-4ff5-a9b0-2de6e4e405c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:15:06.841575 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:15:06.886465 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7151274-3519-4ff5-a9b0-2de6e4e405c0 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:15:06.890153 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a865718-5825-4f4b-b447-a7d58fc95ae1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3e1a2b83f3d4d88ba5084dfaffd22f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:15:06.890805 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a865718-5825-4f4b-b447-a7d58fc95ae1 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:15:06.891096 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a865718-5825-4f4b-b447-a7d58fc95ae1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:15:06.891380 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a865718-5825-4f4b-b447-a7d58fc95ae1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:15:06.901041 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a865718-5825-4f4b-b447-a7d58fc95ae1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:15:06.902494 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a865718-5825-4f4b-b447-a7d58fc95ae1 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:15:06.913048 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1374/2778] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:15:06 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 16:15:16.806462 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=50,keystone:INSERT=3 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:15:16.907627 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=19 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:15:26.270944 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079949cb-5dfc-4737-877a-b81d48830459 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:15:26.336235 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079949cb-5dfc-4737-877a-b81d48830459 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:15:26.342184 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b63373-f36d-4256-958b-cf7ed217e690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:15:26.343059 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b63373-f36d-4256-958b-cf7ed217e690 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:15:26.343348 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b63373-f36d-4256-958b-cf7ed217e690 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:15:26.343651 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b63373-f36d-4256-958b-cf7ed217e690 admin admin] PATH_INFO: `/v3/users/faeec826cf964066bda85d1e20ba0da9` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:15:26.344849 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b63373-f36d-4256-958b-cf7ed217e690 admin admin] RBAC: Authorizing `identity:delete_user(user_id=faeec826cf964066bda85d1e20ba0da9)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:15:26.346785 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b63373-f36d-4256-958b-cf7ed217e690 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:15:26.393481 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-d5b63373-f36d-4256-958b-cf7ed217e690 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'faeec826cf964066bda85d1e20ba0da9'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:15:26.401512 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-d5b63373-f36d-4256-958b-cf7ed217e690 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'faeec826cf964066bda85d1e20ba0da9'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:15:26.413671 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-d5b63373-f36d-4256-958b-cf7ed217e690 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'faeec826cf964066bda85d1e20ba0da9'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:15:26.414334 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-d5b63373-f36d-4256-958b-cf7ed217e690 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'faeec826cf964066bda85d1e20ba0da9'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:15:26.415564 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-d5b63373-f36d-4256-958b-cf7ed217e690 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'faeec826cf964066bda85d1e20ba0da9'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:15:26.428083 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1405/2779] 10.222.0.99 () {58 vars in 1223 bytes} [Thu Jul 14 16:15:26 2022] DELETE /identity/v3/users/faeec826cf964066bda85d1e20ba0da9 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:15:26.432864 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ea7826-7df8-492e-866b-87b1544b4bf8 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:15:26.433349 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ea7826-7df8-492e-866b-87b1544b4bf8 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:15:26.433498 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ea7826-7df8-492e-866b-87b1544b4bf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:15:26.450869 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-c8ea7826-7df8-492e-866b-87b1544b4bf8 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:15:26.452125 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c8ea7826-7df8-492e-866b-87b1544b4bf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:15:26.511764 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1375/2780] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 16:15:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:15:26.521417 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b63373-f36d-4256-958b-cf7ed217e690 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:15:26.522863 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-d5b63373-f36d-4256-958b-cf7ed217e690 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:15:26.583014 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b63373-f36d-4256-958b-cf7ed217e690 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:15:26.587399 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8409cb9d-65ce-49bc-b6e4-f8a382066a05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:15:26.588073 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8409cb9d-65ce-49bc-b6e4-f8a382066a05 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:15:26.588364 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8409cb9d-65ce-49bc-b6e4-f8a382066a05 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:15:26.588643 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8409cb9d-65ce-49bc-b6e4-f8a382066a05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:15:26.597661 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8409cb9d-65ce-49bc-b6e4-f8a382066a05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:15:26.599118 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8409cb9d-65ce-49bc-b6e4-f8a382066a05 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:15:26.608569 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1406/2781] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:15:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:15:26.874837 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ea7826-7df8-492e-866b-87b1544b4bf8 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:15:26.876328 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c8ea7826-7df8-492e-866b-87b1544b4bf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:15:26.892877 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ea7826-7df8-492e-866b-87b1544b4bf8 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:15:26.897724 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bca631-dcca-44e7-8554-7af6a3aff6fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:15:26.898507 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bca631-dcca-44e7-8554-7af6a3aff6fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:15:26.898795 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bca631-dcca-44e7-8554-7af6a3aff6fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:15:26.899233 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bca631-dcca-44e7-8554-7af6a3aff6fc admin admin] PATH_INFO: `/v3/projects/f8226bb5ee7e40e8af9a8029c5128b23` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:15:26.901112 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bca631-dcca-44e7-8554-7af6a3aff6fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8226bb5ee7e40e8af9a8029c5128b23)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:15:26.902941 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bca631-dcca-44e7-8554-7af6a3aff6fc admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:15:26.952731 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-26bca631-dcca-44e7-8554-7af6a3aff6fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8226bb5ee7e40e8af9a8029c5128b23', 'request_id': 'req-26bca631-dcca-44e7-8554-7af6a3aff6fc'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:15:26.958929 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-26bca631-dcca-44e7-8554-7af6a3aff6fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8226bb5ee7e40e8af9a8029c5128b23', 'request_id': 'req-26bca631-dcca-44e7-8554-7af6a3aff6fc'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:15:26.965002 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-26bca631-dcca-44e7-8554-7af6a3aff6fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8226bb5ee7e40e8af9a8029c5128b23', 'request_id': 'req-26bca631-dcca-44e7-8554-7af6a3aff6fc'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:15:26.978548 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-26bca631-dcca-44e7-8554-7af6a3aff6fc admin admin] The token cache is being invalidate because project f8226bb5ee7e40e8af9a8029c5128b23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 16:15:26.978861 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-26bca631-dcca-44e7-8554-7af6a3aff6fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:15:26.979501 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-26bca631-dcca-44e7-8554-7af6a3aff6fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:15:26.981092 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1376/2782] 10.222.0.99 () {58 vars in 1232 bytes} [Thu Jul 14 16:15:26 2022] DELETE /identity/v3/projects/f8226bb5ee7e40e8af9a8029c5128b23 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:15:36.604358 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:15:36.974806 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:17:10.484434 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8409cb9d-65ce-49bc-b6e4-f8a382066a05 service neutron] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:17:10.485838 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-8409cb9d-65ce-49bc-b6e4-f8a382066a05 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:17:10.554950 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8409cb9d-65ce-49bc-b6e4-f8a382066a05 service neutron] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:17:10.605301 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a48dc50-abb7-4c0b-acaa-2538dbefbe4a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3e1a2b83f3d4d88ba5084dfaffd22f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:17:10.605950 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a48dc50-abb7-4c0b-acaa-2538dbefbe4a service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:17:10.606238 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a48dc50-abb7-4c0b-acaa-2538dbefbe4a service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:17:10.606517 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a48dc50-abb7-4c0b-acaa-2538dbefbe4a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:17:10.608339 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-4a48dc50-abb7-4c0b-acaa-2538dbefbe4a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:17:10.700475 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a48dc50-abb7-4c0b-acaa-2538dbefbe4a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:17:10.701916 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a48dc50-abb7-4c0b-acaa-2538dbefbe4a service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:17:10.711908 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1407/2783] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:17:10 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:17:20.707643 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=83 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:18:59.015025 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bca631-dcca-44e7-8554-7af6a3aff6fc admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:18:59.016519 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-26bca631-dcca-44e7-8554-7af6a3aff6fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:18:59.086806 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bca631-dcca-44e7-8554-7af6a3aff6fc admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:18:59.092364 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44338447-378a-43d4-a5a7-61f56433f777 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:18:59.093025 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44338447-378a-43d4-a5a7-61f56433f777 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:18:59.093324 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44338447-378a-43d4-a5a7-61f56433f777 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:18:59.093608 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44338447-378a-43d4-a5a7-61f56433f777 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:18:59.095464 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-44338447-378a-43d4-a5a7-61f56433f777 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:18:59.186299 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44338447-378a-43d4-a5a7-61f56433f777 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:18:59.187795 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44338447-378a-43d4-a5a7-61f56433f777 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:18:59.206485 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1377/2784] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:18:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:19:02.095309 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a48dc50-abb7-4c0b-acaa-2538dbefbe4a service cinder] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:19:02.096803 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-4a48dc50-abb7-4c0b-acaa-2538dbefbe4a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:19:02.183764 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44338447-378a-43d4-a5a7-61f56433f777 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:19:02.186382 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-44338447-378a-43d4-a5a7-61f56433f777 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:19:02.202485 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44338447-378a-43d4-a5a7-61f56433f777 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:19:02.206457 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a48dc50-abb7-4c0b-acaa-2538dbefbe4a service cinder] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:19:02.208837 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ed6c14-153d-4d9c-b1d7-e9f6a6f2f397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:19:02.209723 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ed6c14-153d-4d9c-b1d7-e9f6a6f2f397 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:19:02.210007 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ed6c14-153d-4d9c-b1d7-e9f6a6f2f397 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:19:02.210289 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ed6c14-153d-4d9c-b1d7-e9f6a6f2f397 admin admin] PATH_INFO: `/v3/users/781567471d0f4ca4add4331d405febd4` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:19:02.211510 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ed6c14-153d-4d9c-b1d7-e9f6a6f2f397 admin admin] RBAC: Authorizing `identity:delete_user(user_id=781567471d0f4ca4add4331d405febd4)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:19:02.211825 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8994aae2-a1c7-4ae0-8b36-a6bb23436db4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc157c485f794e968a60d8405c9310ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:19:02.212639 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8994aae2-a1c7-4ae0-8b36-a6bb23436db4 service placement] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:19:02.212926 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8994aae2-a1c7-4ae0-8b36-a6bb23436db4 service placement] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:19:02.213381 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8994aae2-a1c7-4ae0-8b36-a6bb23436db4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:19:02.213568 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ed6c14-153d-4d9c-b1d7-e9f6a6f2f397 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:19:02.215186 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-8994aae2-a1c7-4ae0-8b36-a6bb23436db4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:19:02.227221 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8994aae2-a1c7-4ae0-8b36-a6bb23436db4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:19:02.229379 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8994aae2-a1c7-4ae0-8b36-a6bb23436db4 service placement] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:19:02.239587 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1408/2785] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:19:02 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:19:02.271361 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-18ed6c14-153d-4d9c-b1d7-e9f6a6f2f397 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '781567471d0f4ca4add4331d405febd4'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:19:02.279712 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-18ed6c14-153d-4d9c-b1d7-e9f6a6f2f397 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '781567471d0f4ca4add4331d405febd4'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:19:02.281704 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:19:02.295056 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-18ed6c14-153d-4d9c-b1d7-e9f6a6f2f397 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '781567471d0f4ca4add4331d405febd4'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:19:02.295736 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-18ed6c14-153d-4d9c-b1d7-e9f6a6f2f397 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '781567471d0f4ca4add4331d405febd4'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:19:02.296382 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-18ed6c14-153d-4d9c-b1d7-e9f6a6f2f397 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '781567471d0f4ca4add4331d405febd4'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:19:02.308995 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1378/2786] 10.222.0.99 () {58 vars in 1223 bytes} [Thu Jul 14 16:19:02 2022] DELETE /identity/v3/users/781567471d0f4ca4add4331d405febd4 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:19:02.487748 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8994aae2-a1c7-4ae0-8b36-a6bb23436db4 service placement] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:19:02.498504 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8994aae2-a1c7-4ae0-8b36-a6bb23436db4 service placement] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:19:02.561520 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df99ca05-e854-4473-962d-75d520920d99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:19:02.562476 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df99ca05-e854-4473-962d-75d520920d99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:19:02.562842 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df99ca05-e854-4473-962d-75d520920d99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:19:02.563305 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df99ca05-e854-4473-962d-75d520920d99 admin admin] PATH_INFO: `/v3/projects/636b923a8a58435484086dc8394ccc23` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:19:02.564719 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df99ca05-e854-4473-962d-75d520920d99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=636b923a8a58435484086dc8394ccc23)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:19:02.566650 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df99ca05-e854-4473-962d-75d520920d99 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:19:02.616689 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-df99ca05-e854-4473-962d-75d520920d99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '636b923a8a58435484086dc8394ccc23', 'request_id': 'req-df99ca05-e854-4473-962d-75d520920d99'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:19:02.622873 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-df99ca05-e854-4473-962d-75d520920d99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '636b923a8a58435484086dc8394ccc23', 'request_id': 'req-df99ca05-e854-4473-962d-75d520920d99'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:19:02.628323 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-df99ca05-e854-4473-962d-75d520920d99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '636b923a8a58435484086dc8394ccc23', 'request_id': 'req-df99ca05-e854-4473-962d-75d520920d99'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:19:02.641576 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-df99ca05-e854-4473-962d-75d520920d99 admin admin] The token cache is being invalidate because project 636b923a8a58435484086dc8394ccc23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 16:19:02.641864 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-df99ca05-e854-4473-962d-75d520920d99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:19:02.642470 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-df99ca05-e854-4473-962d-75d520920d99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:19:02.644026 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1409/2787] 10.222.0.99 () {58 vars in 1232 bytes} [Thu Jul 14 16:19:02 2022] DELETE /identity/v3/projects/636b923a8a58435484086dc8394ccc23 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:19:02.650806 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a413a98b-dae8-43cc-8202-b311b73e7fbe None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:19:02.651099 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a413a98b-dae8-43cc-8202-b311b73e7fbe None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:19:02.651381 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a413a98b-dae8-43cc-8202-b311b73e7fbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:19:02.669131 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-a413a98b-dae8-43cc-8202-b311b73e7fbe None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:19:02.670337 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a413a98b-dae8-43cc-8202-b311b73e7fbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:19:02.858037 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1379/2788] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 16:19:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 16:19:02.865837 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d648ab8b-c95b-4711-80bf-b518ef0d2ee9 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:19:02.866127 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d648ab8b-c95b-4711-80bf-b518ef0d2ee9 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:19:02.866408 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d648ab8b-c95b-4711-80bf-b518ef0d2ee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:19:02.886761 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-d648ab8b-c95b-4711-80bf-b518ef0d2ee9 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:19:02.888030 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-d648ab8b-c95b-4711-80bf-b518ef0d2ee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:19:02.895932 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1410/2789] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 16:19:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:19:02.900364 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a413a98b-dae8-43cc-8202-b311b73e7fbe None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:19:02.910578 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a413a98b-dae8-43cc-8202-b311b73e7fbe None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:19:02.915112 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce22ea8-a297-48d2-b769-168e966d295c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:19:02.915912 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce22ea8-a297-48d2-b769-168e966d295c admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:19:02.916344 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce22ea8-a297-48d2-b769-168e966d295c admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:19:02.916632 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce22ea8-a297-48d2-b769-168e966d295c admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:19:02.917358 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce22ea8-a297-48d2-b769-168e966d295c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 16:19:02.917667 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce22ea8-a297-48d2-b769-168e966d295c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:19:02.919255 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce22ea8-a297-48d2-b769-168e966d295c admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:19:02.932013 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1380/2790] 10.222.0.99 () {58 vars in 1152 bytes} [Thu Jul 14 16:19:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 16:19:02.936517 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d648ab8b-c95b-4711-80bf-b518ef0d2ee9 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:19:02.941092 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:19:02.948449 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d648ab8b-c95b-4711-80bf-b518ef0d2ee9 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:19:02.954237 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc208625-b3ff-47c2-8ef3-d9309d1bd39d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:19:02.954933 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc208625-b3ff-47c2-8ef3-d9309d1bd39d admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:19:02.955237 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc208625-b3ff-47c2-8ef3-d9309d1bd39d admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:19:02.955517 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc208625-b3ff-47c2-8ef3-d9309d1bd39d admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:19:02.956250 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc208625-b3ff-47c2-8ef3-d9309d1bd39d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:19:02.958163 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc208625-b3ff-47c2-8ef3-d9309d1bd39d admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:19:02.983198 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1411/2791] 10.222.0.99 () {60 vars in 1152 bytes} [Thu Jul 14 16:19:02 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 16:19:02.987394 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce22ea8-a297-48d2-b769-168e966d295c admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:19:02.998725 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce22ea8-a297-48d2-b769-168e966d295c admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:19:03.053941 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceafd4eb-60be-4692-9f95-a2dc33a0125d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:19:03.054609 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceafd4eb-60be-4692-9f95-a2dc33a0125d admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:19:03.054894 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceafd4eb-60be-4692-9f95-a2dc33a0125d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:19:03.055170 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceafd4eb-60be-4692-9f95-a2dc33a0125d admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:19:03.055881 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceafd4eb-60be-4692-9f95-a2dc33a0125d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:19:03.058511 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceafd4eb-60be-4692-9f95-a2dc33a0125d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:19:03.078845 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1381/2792] 10.222.0.99 () {60 vars in 1143 bytes} [Thu Jul 14 16:19:02 2022] POST /identity/v3/users => generated 293 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 16:19:03.083011 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc208625-b3ff-47c2-8ef3-d9309d1bd39d admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:19:03.093656 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc208625-b3ff-47c2-8ef3-d9309d1bd39d admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:19:03.098217 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f95dea-c401-44fa-8875-7a9e4e70f0b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:19:03.098885 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f95dea-c401-44fa-8875-7a9e4e70f0b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:19:03.099175 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f95dea-c401-44fa-8875-7a9e4e70f0b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:19:03.099457 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f95dea-c401-44fa-8875-7a9e4e70f0b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:19:03.100231 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f95dea-c401-44fa-8875-7a9e4e70f0b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:19:03.101819 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f95dea-c401-44fa-8875-7a9e4e70f0b2 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:19:03.112178 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1412/2793] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 16:19:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:19:03.116059 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceafd4eb-60be-4692-9f95-a2dc33a0125d admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:19:03.127789 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceafd4eb-60be-4692-9f95-a2dc33a0125d admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:19:03.132370 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce4dfe2-4f5e-490c-97ff-689d39ddcff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:19:03.133032 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce4dfe2-4f5e-490c-97ff-689d39ddcff6 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:19:03.133319 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce4dfe2-4f5e-490c-97ff-689d39ddcff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:19:03.133600 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce4dfe2-4f5e-490c-97ff-689d39ddcff6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:19:03.134381 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce4dfe2-4f5e-490c-97ff-689d39ddcff6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:19:03.136512 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce4dfe2-4f5e-490c-97ff-689d39ddcff6 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:19:03.146379 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1382/2794] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 16:19:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 16:19:03.151023 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f95dea-c401-44fa-8875-7a9e4e70f0b2 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:19:03.161228 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f95dea-c401-44fa-8875-7a9e4e70f0b2 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:19:03.166547 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ee882f-2dd5-4419-a75e-7a333e80984c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:19:03.167467 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ee882f-2dd5-4419-a75e-7a333e80984c admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:19:03.167797 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ee882f-2dd5-4419-a75e-7a333e80984c admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:19:03.168086 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ee882f-2dd5-4419-a75e-7a333e80984c admin admin] PATH_INFO: `/v3/projects/c09479647ca24b63b744f3dd55111cb3/users/b5dbfe771fbe4372951006b71082391a/roles/b8c98fddc0fd4436b577fd1d3361bc38` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:19:03.183394 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ee882f-2dd5-4419-a75e-7a333e80984c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c09479647ca24b63b744f3dd55111cb3, user_id=b5dbfe771fbe4372951006b71082391a, role_id=b8c98fddc0fd4436b577fd1d3361bc38)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:19:03.186088 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ee882f-2dd5-4419-a75e-7a333e80984c admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:19:03.199364 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1413/2795] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 16:19:03 2022] PUT /identity/v3/projects/c09479647ca24b63b744f3dd55111cb3/users/b5dbfe771fbe4372951006b71082391a/roles/b8c98fddc0fd4436b577fd1d3361bc38 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:19:03.204435 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0244c04b-fafd-4861-b5cc-2303e238e54a None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:19:03.204991 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0244c04b-fafd-4861-b5cc-2303e238e54a None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:19:03.204991 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0244c04b-fafd-4861-b5cc-2303e238e54a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:19:03.222548 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-0244c04b-fafd-4861-b5cc-2303e238e54a None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:19:03.223815 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0244c04b-fafd-4861-b5cc-2303e238e54a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:19:03.257812 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:19:03.286104 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1383/2796] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 16:19:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:19:03.296431 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ee882f-2dd5-4419-a75e-7a333e80984c admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:19:03.298150 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-55ee882f-2dd5-4419-a75e-7a333e80984c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:19:03.356929 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ee882f-2dd5-4419-a75e-7a333e80984c admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:19:03.360636 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8a5885-9496-425b-a03c-a002af91ede0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:19:03.361291 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8a5885-9496-425b-a03c-a002af91ede0 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:19:03.361584 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8a5885-9496-425b-a03c-a002af91ede0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:19:03.361865 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8a5885-9496-425b-a03c-a002af91ede0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:19:03.371757 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8a5885-9496-425b-a03c-a002af91ede0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:19:03.373231 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8a5885-9496-425b-a03c-a002af91ede0 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:19:03.383097 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1414/2797] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:19:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:19:07.943820 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177e20eb-8498-4b3a-a833-d7d346ba065b None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:19:07.944557 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177e20eb-8498-4b3a-a833-d7d346ba065b None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:19:07.944557 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177e20eb-8498-4b3a-a833-d7d346ba065b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:19:07.980303 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-177e20eb-8498-4b3a-a833-d7d346ba065b None None] MFA Rules not processed for user `b5dbfe771fbe4372951006b71082391a`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:19:07.981839 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-177e20eb-8498-4b3a-a833-d7d346ba065b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:19:08.060649 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1384/2798] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 16:19:07 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:19:08.077869 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8a5885-9496-425b-a03c-a002af91ede0 service neutron] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:19:08.079586 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-2a8a5885-9496-425b-a03c-a002af91ede0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:19:08.176468 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8a5885-9496-425b-a03c-a002af91ede0 service neutron] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:19:08.182466 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa769f48-47eb-4c99-8fc1-3311471811c5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9b1c2622934c5185f3b5f1ea623bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:19:08.183131 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa769f48-47eb-4c99-8fc1-3311471811c5 service nova] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:19:08.183422 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa769f48-47eb-4c99-8fc1-3311471811c5 service nova] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:19:08.183740 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa769f48-47eb-4c99-8fc1-3311471811c5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:19:08.192778 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa769f48-47eb-4c99-8fc1-3311471811c5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:19:08.194320 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa769f48-47eb-4c99-8fc1-3311471811c5 service nova] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:19:08.206286 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1415/2799] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:19:08 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:19:10.684294 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ea9277-1647-4e4c-9830-43e320268143 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:19:10.685207 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ea9277-1647-4e4c-9830-43e320268143 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:19:10.685504 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ea9277-1647-4e4c-9830-43e320268143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:19:10.708575 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-25ea9277-1647-4e4c-9830-43e320268143 None None] MFA Rules not processed for user `b5dbfe771fbe4372951006b71082391a`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:19:10.709877 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-25ea9277-1647-4e4c-9830-43e320268143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:19:10.719865 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1385/2800] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 16:19:10 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:19:10.765711 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa769f48-47eb-4c99-8fc1-3311471811c5 service nova] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:19:10.768225 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-fa769f48-47eb-4c99-8fc1-3311471811c5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:19:10.800205 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:19:10.897267 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa769f48-47eb-4c99-8fc1-3311471811c5 service nova] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:19:10.901285 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c0e386-7d77-4621-a649-79801ca9a253 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605e230d50ce44629f554c95b2f79ead', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:19:10.902003 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c0e386-7d77-4621-a649-79801ca9a253 service glance] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:19:10.902295 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c0e386-7d77-4621-a649-79801ca9a253 service glance] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:19:10.902577 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c0e386-7d77-4621-a649-79801ca9a253 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:19:10.905048 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-36c0e386-7d77-4621-a649-79801ca9a253 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:19:10.924966 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c0e386-7d77-4621-a649-79801ca9a253 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:19:10.927074 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c0e386-7d77-4621-a649-79801ca9a253 service glance] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:19:10.941509 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1416/2801] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:19:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:19:11.311431 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ea9277-1647-4e4c-9830-43e320268143 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:19:11.312725 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-25ea9277-1647-4e4c-9830-43e320268143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:19:11.325669 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ea9277-1647-4e4c-9830-43e320268143 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:19:11.329280 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776df6be-2b70-4091-ae71-a0d74fc0f3e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:19:11.329935 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776df6be-2b70-4091-ae71-a0d74fc0f3e5 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:19:11.330267 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776df6be-2b70-4091-ae71-a0d74fc0f3e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:19:11.330495 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776df6be-2b70-4091-ae71-a0d74fc0f3e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:19:11.332374 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-776df6be-2b70-4091-ae71-a0d74fc0f3e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:19:11.343957 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776df6be-2b70-4091-ae71-a0d74fc0f3e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:19:11.345412 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776df6be-2b70-4091-ae71-a0d74fc0f3e5 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:19:11.356395 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1386/2802] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:19:11 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:19:11.715112 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c0e386-7d77-4621-a649-79801ca9a253 service glance] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:19:11.716559 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-36c0e386-7d77-4621-a649-79801ca9a253 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:19:11.727180 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776df6be-2b70-4091-ae71-a0d74fc0f3e5 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:19:11.745837 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776df6be-2b70-4091-ae71-a0d74fc0f3e5 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:19:11.752301 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9377b7f-0910-4c34-90a4-0c085ab648d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:19:11.752973 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9377b7f-0910-4c34-90a4-0c085ab648d7 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:19:11.753266 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9377b7f-0910-4c34-90a4-0c085ab648d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:19:11.753544 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9377b7f-0910-4c34-90a4-0c085ab648d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:19:11.755642 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f9377b7f-0910-4c34-90a4-0c085ab648d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:19:11.771032 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9377b7f-0910-4c34-90a4-0c085ab648d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:19:11.772520 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9377b7f-0910-4c34-90a4-0c085ab648d7 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:19:11.794275 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1387/2803] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:19:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 16:19:11.810366 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c0e386-7d77-4621-a649-79801ca9a253 service glance] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:19:11.816108 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd806a67-0bf5-4be9-859c-22ce6f1a0018 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc157c485f794e968a60d8405c9310ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:19:11.816782 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd806a67-0bf5-4be9-859c-22ce6f1a0018 service placement] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:19:11.817062 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd806a67-0bf5-4be9-859c-22ce6f1a0018 service placement] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:19:11.817519 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd806a67-0bf5-4be9-859c-22ce6f1a0018 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:19:11.819506 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-bd806a67-0bf5-4be9-859c-22ce6f1a0018 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:19:11.834382 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd806a67-0bf5-4be9-859c-22ce6f1a0018 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:19:11.836022 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd806a67-0bf5-4be9-859c-22ce6f1a0018 service placement] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:19:11.848534 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1417/2804] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:19:11 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:19:13.722275 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9377b7f-0910-4c34-90a4-0c085ab648d7 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:19:13.732054 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9377b7f-0910-4c34-90a4-0c085ab648d7 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:19:13.736000 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a3b57a-76f2-420f-9de7-4514cbc0ad77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:19:13.736676 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a3b57a-76f2-420f-9de7-4514cbc0ad77 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:19:13.736962 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a3b57a-76f2-420f-9de7-4514cbc0ad77 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:19:13.737237 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a3b57a-76f2-420f-9de7-4514cbc0ad77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:19:13.739181 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-31a3b57a-76f2-420f-9de7-4514cbc0ad77 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:19:13.749735 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a3b57a-76f2-420f-9de7-4514cbc0ad77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:19:13.751301 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a3b57a-76f2-420f-9de7-4514cbc0ad77 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:19:13.761161 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1388/2805] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:19:13 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:19:14.419694 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd806a67-0bf5-4be9-859c-22ce6f1a0018 service placement] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:19:14.421324 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-bd806a67-0bf5-4be9-859c-22ce6f1a0018 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:19:14.441226 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd806a67-0bf5-4be9-859c-22ce6f1a0018 service placement] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:19:14.448393 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c671662-65a7-4e1e-8660-46b5c7a4ae1f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9b1c2622934c5185f3b5f1ea623bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:19:14.449623 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c671662-65a7-4e1e-8660-46b5c7a4ae1f service nova] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:19:14.449990 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c671662-65a7-4e1e-8660-46b5c7a4ae1f service nova] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:19:14.450334 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c671662-65a7-4e1e-8660-46b5c7a4ae1f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:19:14.452449 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-9c671662-65a7-4e1e-8660-46b5c7a4ae1f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:19:14.467776 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c671662-65a7-4e1e-8660-46b5c7a4ae1f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:19:14.469440 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c671662-65a7-4e1e-8660-46b5c7a4ae1f service nova] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:19:14.484258 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1418/2806] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:19:14 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:19:19.572976 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a3b57a-76f2-420f-9de7-4514cbc0ad77 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:19:19.586766 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a3b57a-76f2-420f-9de7-4514cbc0ad77 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:19:19.592781 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209f283a-b6bb-4d03-af30-0aac9096ad72 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9b1c2622934c5185f3b5f1ea623bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:19:19.593436 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209f283a-b6bb-4d03-af30-0aac9096ad72 service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:19:19.593719 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209f283a-b6bb-4d03-af30-0aac9096ad72 service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:19:19.593993 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209f283a-b6bb-4d03-af30-0aac9096ad72 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:19:19.595935 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-209f283a-b6bb-4d03-af30-0aac9096ad72 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:19:19.609224 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209f283a-b6bb-4d03-af30-0aac9096ad72 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:19:19.610866 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209f283a-b6bb-4d03-af30-0aac9096ad72 service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:19:19.622112 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1389/2807] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:19:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 16:19:24.477251 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=65 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:19:29.617832 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=68 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:20:05.635161 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c671662-65a7-4e1e-8660-46b5c7a4ae1f service nova] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:20:05.647302 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c671662-65a7-4e1e-8660-46b5c7a4ae1f service nova] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:20:05.656436 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdaa9c3-d0a8-4f3b-9bb6-bb6bee47d139 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc157c485f794e968a60d8405c9310ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:20:05.657066 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdaa9c3-d0a8-4f3b-9bb6-bb6bee47d139 service placement] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:20:05.657357 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdaa9c3-d0a8-4f3b-9bb6-bb6bee47d139 service placement] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:20:05.657642 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdaa9c3-d0a8-4f3b-9bb6-bb6bee47d139 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:20:05.659436 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-5fdaa9c3-d0a8-4f3b-9bb6-bb6bee47d139 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:20:05.671820 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdaa9c3-d0a8-4f3b-9bb6-bb6bee47d139 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:20:05.673468 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdaa9c3-d0a8-4f3b-9bb6-bb6bee47d139 service placement] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:20:05.683893 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1419/2808] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:20:05 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:20:15.679511 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=6 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:20:17.868905 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970d5cbd-5775-4045-8876-530d433fe0cd None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:20:17.869452 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970d5cbd-5775-4045-8876-530d433fe0cd None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:20:17.869569 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970d5cbd-5775-4045-8876-530d433fe0cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:20:17.887731 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-970d5cbd-5775-4045-8876-530d433fe0cd None None] MFA Rules not processed for user `b5dbfe771fbe4372951006b71082391a`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:20:17.888946 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-970d5cbd-5775-4045-8876-530d433fe0cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:20:17.896533 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1390/2809] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 16:20:17 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:20:18.207001 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdaa9c3-d0a8-4f3b-9bb6-bb6bee47d139 service placement] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:20:18.216212 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdaa9c3-d0a8-4f3b-9bb6-bb6bee47d139 service placement] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:20:18.220528 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8592a5bf-b131-4049-9394-c7fb6b1364f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:20:18.221180 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8592a5bf-b131-4049-9394-c7fb6b1364f8 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:20:18.221466 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8592a5bf-b131-4049-9394-c7fb6b1364f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:20:18.221751 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8592a5bf-b131-4049-9394-c7fb6b1364f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:20:18.223518 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-8592a5bf-b131-4049-9394-c7fb6b1364f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:20:18.233254 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8592a5bf-b131-4049-9394-c7fb6b1364f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:20:18.234880 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8592a5bf-b131-4049-9394-c7fb6b1364f8 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:20:18.242879 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1420/2810] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:20:18 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:20:27.882548 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:20:28.239290 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=6 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:20:29.024358 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970d5cbd-5775-4045-8876-530d433fe0cd None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:20:29.038225 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970d5cbd-5775-4045-8876-530d433fe0cd None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:20:29.043686 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc94595-68fc-441b-9f42-9839159cb2ff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9b1c2622934c5185f3b5f1ea623bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:20:29.044345 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc94595-68fc-441b-9f42-9839159cb2ff service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:20:29.044635 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc94595-68fc-441b-9f42-9839159cb2ff service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:20:29.044919 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc94595-68fc-441b-9f42-9839159cb2ff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:20:29.048205 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-fdc94595-68fc-441b-9f42-9839159cb2ff service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:20:29.074715 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc94595-68fc-441b-9f42-9839159cb2ff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:20:29.076272 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc94595-68fc-441b-9f42-9839159cb2ff service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:20:29.088673 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1391/2811] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:20:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:20:39.083979 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=11 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:21:25.197953 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc283da-e99b-48f8-af3b-8591bcab4a6e None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:21:25.198571 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc283da-e99b-48f8-af3b-8591bcab4a6e None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:21:25.198571 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc283da-e99b-48f8-af3b-8591bcab4a6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:21:25.216611 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-4cc283da-e99b-48f8-af3b-8591bcab4a6e None None] MFA Rules not processed for user `b5dbfe771fbe4372951006b71082391a`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:21:25.217833 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-4cc283da-e99b-48f8-af3b-8591bcab4a6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:21:25.224980 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1421/2812] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 16:21:25 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:21:35.210827 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=3 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:22:36.699197 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a19438f-8277-4303-86a5-1513c54895c0 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:22:36.699705 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a19438f-8277-4303-86a5-1513c54895c0 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:22:36.699787 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a19438f-8277-4303-86a5-1513c54895c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:22:36.719729 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-9a19438f-8277-4303-86a5-1513c54895c0 None None] MFA Rules not processed for user `b5dbfe771fbe4372951006b71082391a`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:22:36.720956 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-9a19438f-8277-4303-86a5-1513c54895c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:22:36.728921 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1392/2813] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 16:22:36 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:22:46.714540 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:23:51.201322 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc283da-e99b-48f8-af3b-8591bcab4a6e None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:23:51.216728 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc283da-e99b-48f8-af3b-8591bcab4a6e None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:23:51.221744 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1385e9a8-d63c-4a6a-9375-256e3f023093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:23:51.222778 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1385e9a8-d63c-4a6a-9375-256e3f023093 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:23:51.223167 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1385e9a8-d63c-4a6a-9375-256e3f023093 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:23:51.223523 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1385e9a8-d63c-4a6a-9375-256e3f023093 admin admin] PATH_INFO: `/v3/users/b5dbfe771fbe4372951006b71082391a` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:23:51.224943 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1385e9a8-d63c-4a6a-9375-256e3f023093 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5dbfe771fbe4372951006b71082391a)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:23:51.227272 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1385e9a8-d63c-4a6a-9375-256e3f023093 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:23:51.283104 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-1385e9a8-d63c-4a6a-9375-256e3f023093 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5dbfe771fbe4372951006b71082391a'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:23:51.290771 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-1385e9a8-d63c-4a6a-9375-256e3f023093 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5dbfe771fbe4372951006b71082391a'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:23:51.302988 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-1385e9a8-d63c-4a6a-9375-256e3f023093 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5dbfe771fbe4372951006b71082391a'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:23:51.303639 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-1385e9a8-d63c-4a6a-9375-256e3f023093 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5dbfe771fbe4372951006b71082391a'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:23:51.305230 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-1385e9a8-d63c-4a6a-9375-256e3f023093 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5dbfe771fbe4372951006b71082391a'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:23:51.317745 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1422/2814] 10.222.0.99 () {58 vars in 1223 bytes} [Thu Jul 14 16:23:51 2022] DELETE /identity/v3/users/b5dbfe771fbe4372951006b71082391a => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:23:51.501400 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a19438f-8277-4303-86a5-1513c54895c0 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:23:51.513798 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a19438f-8277-4303-86a5-1513c54895c0 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:23:51.574013 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5723772c-a5cb-40da-bfe8-b1a6f0848e3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:23:51.574871 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5723772c-a5cb-40da-bfe8-b1a6f0848e3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:23:51.575156 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5723772c-a5cb-40da-bfe8-b1a6f0848e3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:23:51.575434 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5723772c-a5cb-40da-bfe8-b1a6f0848e3d admin admin] PATH_INFO: `/v3/projects/c09479647ca24b63b744f3dd55111cb3` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:23:51.576641 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5723772c-a5cb-40da-bfe8-b1a6f0848e3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c09479647ca24b63b744f3dd55111cb3)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:23:51.578752 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5723772c-a5cb-40da-bfe8-b1a6f0848e3d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:23:51.642518 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5723772c-a5cb-40da-bfe8-b1a6f0848e3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c09479647ca24b63b744f3dd55111cb3', 'request_id': 'req-5723772c-a5cb-40da-bfe8-b1a6f0848e3d'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:23:51.649528 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5723772c-a5cb-40da-bfe8-b1a6f0848e3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c09479647ca24b63b744f3dd55111cb3', 'request_id': 'req-5723772c-a5cb-40da-bfe8-b1a6f0848e3d'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:23:51.655583 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5723772c-a5cb-40da-bfe8-b1a6f0848e3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c09479647ca24b63b744f3dd55111cb3', 'request_id': 'req-5723772c-a5cb-40da-bfe8-b1a6f0848e3d'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:23:51.669110 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5723772c-a5cb-40da-bfe8-b1a6f0848e3d admin admin] The token cache is being invalidate because project c09479647ca24b63b744f3dd55111cb3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 16:23:51.669400 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5723772c-a5cb-40da-bfe8-b1a6f0848e3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:23:51.670083 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5723772c-a5cb-40da-bfe8-b1a6f0848e3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:23:51.671576 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1393/2815] 10.222.0.99 () {58 vars in 1232 bytes} [Thu Jul 14 16:23:51 2022] DELETE /identity/v3/projects/c09479647ca24b63b744f3dd55111cb3 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:23:51.676683 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e514228-ed5b-43d2-be50-15f1b0d2eba7 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:23:51.676987 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e514228-ed5b-43d2-be50-15f1b0d2eba7 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:23:51.677279 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e514228-ed5b-43d2-be50-15f1b0d2eba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:23:51.736764 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-9e514228-ed5b-43d2-be50-15f1b0d2eba7 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:23:51.737989 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-9e514228-ed5b-43d2-be50-15f1b0d2eba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:23:51.798524 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1423/2816] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 16:23:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:23:51.808726 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4b90c0-0d45-4148-b421-6e110f5b5130 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:23:51.809017 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4b90c0-0d45-4148-b421-6e110f5b5130 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:23:51.809301 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4b90c0-0d45-4148-b421-6e110f5b5130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:23:51.827954 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-8f4b90c0-0d45-4148-b421-6e110f5b5130 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:23:51.829171 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-8f4b90c0-0d45-4148-b421-6e110f5b5130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:23:51.837491 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1394/2817] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 16:23:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:23:51.841866 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e514228-ed5b-43d2-be50-15f1b0d2eba7 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:23:51.852094 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e514228-ed5b-43d2-be50-15f1b0d2eba7 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:23:51.856713 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e604d6-53bc-4d1a-8f6b-fa1877e9448b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:23:51.857364 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e604d6-53bc-4d1a-8f6b-fa1877e9448b admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:23:51.857647 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e604d6-53bc-4d1a-8f6b-fa1877e9448b admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:23:51.857923 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e604d6-53bc-4d1a-8f6b-fa1877e9448b admin admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:23:51.858655 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e604d6-53bc-4d1a-8f6b-fa1877e9448b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 16:23:51.858961 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e604d6-53bc-4d1a-8f6b-fa1877e9448b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:23:51.860588 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e604d6-53bc-4d1a-8f6b-fa1877e9448b admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:23:51.872860 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1424/2818] 10.222.0.99 () {58 vars in 1152 bytes} [Thu Jul 14 16:23:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 16:23:51.877284 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4b90c0-0d45-4148-b421-6e110f5b5130 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:23:51.886505 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4b90c0-0d45-4148-b421-6e110f5b5130 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:23:51.891610 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840fe72e-106d-4a47-848f-7ca7ce3f9019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:23:51.892279 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840fe72e-106d-4a47-848f-7ca7ce3f9019 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:23:51.892568 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840fe72e-106d-4a47-848f-7ca7ce3f9019 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:23:51.892846 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840fe72e-106d-4a47-848f-7ca7ce3f9019 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:23:51.893567 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840fe72e-106d-4a47-848f-7ca7ce3f9019 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:23:51.895448 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840fe72e-106d-4a47-848f-7ca7ce3f9019 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:23:51.931455 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1395/2819] 10.222.0.99 () {60 vars in 1152 bytes} [Thu Jul 14 16:23:51 2022] POST /identity/v3/projects => generated 385 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 16:23:51.936017 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e604d6-53bc-4d1a-8f6b-fa1877e9448b admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:23:51.945983 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e604d6-53bc-4d1a-8f6b-fa1877e9448b admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:23:51.973459 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:23:52.009365 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26d059a-79e4-496e-9c55-517aee883db4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:23:52.010042 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26d059a-79e4-496e-9c55-517aee883db4 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:23:52.010332 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26d059a-79e4-496e-9c55-517aee883db4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:23:52.010616 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26d059a-79e4-496e-9c55-517aee883db4 admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:23:52.011325 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26d059a-79e4-496e-9c55-517aee883db4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:23:52.013303 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26d059a-79e4-496e-9c55-517aee883db4 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:23:52.034666 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1425/2820] 10.222.0.99 () {60 vars in 1143 bytes} [Thu Jul 14 16:23:51 2022] POST /identity/v3/users => generated 293 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 16:23:52.037686 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840fe72e-106d-4a47-848f-7ca7ce3f9019 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:23:52.049164 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840fe72e-106d-4a47-848f-7ca7ce3f9019 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:23:52.053753 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7709a33-5281-4985-abc9-4e5f7e0a53b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:23:52.054490 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7709a33-5281-4985-abc9-4e5f7e0a53b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:23:52.054779 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7709a33-5281-4985-abc9-4e5f7e0a53b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:23:52.055061 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7709a33-5281-4985-abc9-4e5f7e0a53b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:23:52.055839 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7709a33-5281-4985-abc9-4e5f7e0a53b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:23:52.057678 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7709a33-5281-4985-abc9-4e5f7e0a53b0 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:23:52.067896 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1396/2821] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 16:23:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:23:52.072007 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26d059a-79e4-496e-9c55-517aee883db4 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:23:52.082276 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26d059a-79e4-496e-9c55-517aee883db4 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:23:52.087382 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f63f43d-7bf5-42d4-a564-326c4b64bbaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:23:52.088074 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f63f43d-7bf5-42d4-a564-326c4b64bbaa admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:23:52.088362 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f63f43d-7bf5-42d4-a564-326c4b64bbaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:23:52.088636 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f63f43d-7bf5-42d4-a564-326c4b64bbaa admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:23:52.089363 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f63f43d-7bf5-42d4-a564-326c4b64bbaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:23:52.090957 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f63f43d-7bf5-42d4-a564-326c4b64bbaa admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:23:52.101645 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1426/2822] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 16:23:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:23:52.105849 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7709a33-5281-4985-abc9-4e5f7e0a53b0 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:23:52.115788 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7709a33-5281-4985-abc9-4e5f7e0a53b0 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:23:52.120527 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e9c8c3-b3a8-4ca2-a3da-58d93621c0db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:23:52.121395 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e9c8c3-b3a8-4ca2-a3da-58d93621c0db admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:23:52.121701 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e9c8c3-b3a8-4ca2-a3da-58d93621c0db admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:23:52.121938 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e9c8c3-b3a8-4ca2-a3da-58d93621c0db admin admin] PATH_INFO: `/v3/projects/e03b6e4c45f640b79b15d2064012c5f9/users/85527d6a8d1e422cac5c907dc699a6ee/roles/b8c98fddc0fd4436b577fd1d3361bc38` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:23:52.136716 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e9c8c3-b3a8-4ca2-a3da-58d93621c0db admin admin] RBAC: Authorizing `identity:create_grant(project_id=e03b6e4c45f640b79b15d2064012c5f9, user_id=85527d6a8d1e422cac5c907dc699a6ee, role_id=b8c98fddc0fd4436b577fd1d3361bc38)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:23:52.139358 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e9c8c3-b3a8-4ca2-a3da-58d93621c0db admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:23:52.153242 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1397/2823] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 16:23:52 2022] PUT /identity/v3/projects/e03b6e4c45f640b79b15d2064012c5f9/users/85527d6a8d1e422cac5c907dc699a6ee/roles/b8c98fddc0fd4436b577fd1d3361bc38 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 16:23:52.163094 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a0f2b5-7ad8-4b41-abbf-37c21042da31 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:23:52.164009 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a0f2b5-7ad8-4b41-abbf-37c21042da31 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:23:52.164300 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a0f2b5-7ad8-4b41-abbf-37c21042da31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:23:52.190910 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-08a0f2b5-7ad8-4b41-abbf-37c21042da31 None None] MFA Rules not processed for user `85527d6a8d1e422cac5c907dc699a6ee`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:23:52.192091 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-08a0f2b5-7ad8-4b41-abbf-37c21042da31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:23:52.256002 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1427/2824] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 16:23:52 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:23:52.267798 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e9c8c3-b3a8-4ca2-a3da-58d93621c0db admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:23:52.269538 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-05e9c8c3-b3a8-4ca2-a3da-58d93621c0db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:23:52.290248 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:23:52.348359 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e9c8c3-b3a8-4ca2-a3da-58d93621c0db admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:23:52.354597 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae1a73a-5904-4bd5-810f-7db3e292c86a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9b1c2622934c5185f3b5f1ea623bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:23:52.355272 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae1a73a-5904-4bd5-810f-7db3e292c86a service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:23:52.355644 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae1a73a-5904-4bd5-810f-7db3e292c86a service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:23:52.355942 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae1a73a-5904-4bd5-810f-7db3e292c86a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:23:52.363792 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae1a73a-5904-4bd5-810f-7db3e292c86a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:23:52.365361 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae1a73a-5904-4bd5-810f-7db3e292c86a service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:23:52.375107 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1398/2825] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:23:52 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:24:00.876952 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a0f2b5-7ad8-4b41-abbf-37c21042da31 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:24:00.941820 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a0f2b5-7ad8-4b41-abbf-37c21042da31 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:24:00.947061 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a01111-7ce5-45c6-924e-6e2d69b36e64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:24:00.947964 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a01111-7ce5-45c6-924e-6e2d69b36e64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:24:00.948248 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a01111-7ce5-45c6-924e-6e2d69b36e64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:24:00.948524 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a01111-7ce5-45c6-924e-6e2d69b36e64 admin admin] PATH_INFO: `/v3/users/85527d6a8d1e422cac5c907dc699a6ee` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:24:00.949811 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a01111-7ce5-45c6-924e-6e2d69b36e64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=85527d6a8d1e422cac5c907dc699a6ee)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:24:00.951841 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a01111-7ce5-45c6-924e-6e2d69b36e64 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:24:00.995031 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:24:01.002329 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-d0a01111-7ce5-45c6-924e-6e2d69b36e64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85527d6a8d1e422cac5c907dc699a6ee'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:24:01.011952 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-d0a01111-7ce5-45c6-924e-6e2d69b36e64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85527d6a8d1e422cac5c907dc699a6ee'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:24:01.024608 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-d0a01111-7ce5-45c6-924e-6e2d69b36e64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85527d6a8d1e422cac5c907dc699a6ee'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:24:01.025244 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-d0a01111-7ce5-45c6-924e-6e2d69b36e64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85527d6a8d1e422cac5c907dc699a6ee'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:24:01.025838 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-d0a01111-7ce5-45c6-924e-6e2d69b36e64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85527d6a8d1e422cac5c907dc699a6ee'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:24:01.037794 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1428/2826] 10.222.0.99 () {58 vars in 1223 bytes} [Thu Jul 14 16:24:00 2022] DELETE /identity/v3/users/85527d6a8d1e422cac5c907dc699a6ee => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:24:01.042546 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea307b44-865f-4246-ad16-19f54513b758 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:24:01.043031 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea307b44-865f-4246-ad16-19f54513b758 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:24:01.043184 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea307b44-865f-4246-ad16-19f54513b758 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:24:01.061521 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-ea307b44-865f-4246-ad16-19f54513b758 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:24:01.062768 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ea307b44-865f-4246-ad16-19f54513b758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:24:01.121754 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1399/2827] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 16:24:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 16:24:01.132368 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a01111-7ce5-45c6-924e-6e2d69b36e64 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:24:01.134006 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-d0a01111-7ce5-45c6-924e-6e2d69b36e64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:24:01.207817 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a01111-7ce5-45c6-924e-6e2d69b36e64 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:24:01.211644 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734fd379-8021-42d7-88e0-61244ea4f830 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:24:01.212299 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734fd379-8021-42d7-88e0-61244ea4f830 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:24:01.212588 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734fd379-8021-42d7-88e0-61244ea4f830 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:24:01.212870 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734fd379-8021-42d7-88e0-61244ea4f830 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:24:01.221837 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734fd379-8021-42d7-88e0-61244ea4f830 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:24:01.223270 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734fd379-8021-42d7-88e0-61244ea4f830 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:24:01.232957 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1429/2828] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:24:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:24:01.512814 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea307b44-865f-4246-ad16-19f54513b758 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:24:01.514247 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ea307b44-865f-4246-ad16-19f54513b758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:24:01.526515 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea307b44-865f-4246-ad16-19f54513b758 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:24:01.531294 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18dc3d6-d7f7-406e-9b43-f94968801c2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:24:01.532214 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18dc3d6-d7f7-406e-9b43-f94968801c2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:24:01.532506 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18dc3d6-d7f7-406e-9b43-f94968801c2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:24:01.532787 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18dc3d6-d7f7-406e-9b43-f94968801c2c admin admin] PATH_INFO: `/v3/projects/e03b6e4c45f640b79b15d2064012c5f9` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:24:01.533981 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18dc3d6-d7f7-406e-9b43-f94968801c2c admin admin] RBAC: Authorizing `identity:delete_project(project_id=e03b6e4c45f640b79b15d2064012c5f9)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:24:01.535945 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18dc3d6-d7f7-406e-9b43-f94968801c2c admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:24:01.583970 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d18dc3d6-d7f7-406e-9b43-f94968801c2c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e03b6e4c45f640b79b15d2064012c5f9', 'request_id': 'req-d18dc3d6-d7f7-406e-9b43-f94968801c2c'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:24:01.590125 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d18dc3d6-d7f7-406e-9b43-f94968801c2c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e03b6e4c45f640b79b15d2064012c5f9', 'request_id': 'req-d18dc3d6-d7f7-406e-9b43-f94968801c2c'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:24:01.595562 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d18dc3d6-d7f7-406e-9b43-f94968801c2c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e03b6e4c45f640b79b15d2064012c5f9', 'request_id': 'req-d18dc3d6-d7f7-406e-9b43-f94968801c2c'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:24:01.608743 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d18dc3d6-d7f7-406e-9b43-f94968801c2c admin admin] The token cache is being invalidate because project e03b6e4c45f640b79b15d2064012c5f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 16:24:01.609039 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d18dc3d6-d7f7-406e-9b43-f94968801c2c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:24:01.609662 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d18dc3d6-d7f7-406e-9b43-f94968801c2c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:24:01.611238 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1400/2829] 10.222.0.99 () {58 vars in 1232 bytes} [Thu Jul 14 16:24:01 2022] DELETE /identity/v3/projects/e03b6e4c45f640b79b15d2064012c5f9 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:24:01.616931 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d5ba73-cfa6-4d6e-a9db-5ce0a07a7c49 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:24:01.617536 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d5ba73-cfa6-4d6e-a9db-5ce0a07a7c49 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:24:01.617536 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d5ba73-cfa6-4d6e-a9db-5ce0a07a7c49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:24:01.635211 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-b2d5ba73-cfa6-4d6e-a9db-5ce0a07a7c49 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:24:01.636456 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-b2d5ba73-cfa6-4d6e-a9db-5ce0a07a7c49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:24:01.695910 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1430/2830] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 16:24:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:24:01.703763 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f42f7f-3980-42d2-afd3-bd9c9787719c None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:24:01.704062 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f42f7f-3980-42d2-afd3-bd9c9787719c None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:24:01.704345 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f42f7f-3980-42d2-afd3-bd9c9787719c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:24:01.723314 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-08f42f7f-3980-42d2-afd3-bd9c9787719c None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:24:01.724658 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-08f42f7f-3980-42d2-afd3-bd9c9787719c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:24:01.733468 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1401/2831] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 16:24:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:24:01.737823 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d5ba73-cfa6-4d6e-a9db-5ce0a07a7c49 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:24:01.747761 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d5ba73-cfa6-4d6e-a9db-5ce0a07a7c49 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:24:01.752348 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbe5414-012a-4447-840a-3cba42ab6d8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:24:01.753039 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbe5414-012a-4447-840a-3cba42ab6d8f admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:24:01.753332 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbe5414-012a-4447-840a-3cba42ab6d8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:24:01.753616 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbe5414-012a-4447-840a-3cba42ab6d8f admin admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:24:01.754346 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbe5414-012a-4447-840a-3cba42ab6d8f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 16:24:01.754660 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbe5414-012a-4447-840a-3cba42ab6d8f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:24:01.756308 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbe5414-012a-4447-840a-3cba42ab6d8f admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:24:01.768194 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1431/2832] 10.222.0.99 () {58 vars in 1152 bytes} [Thu Jul 14 16:24:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 16:24:01.773048 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f42f7f-3980-42d2-afd3-bd9c9787719c None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:24:01.782329 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f42f7f-3980-42d2-afd3-bd9c9787719c None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:24:01.787370 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f28b9d-eb0e-489a-a913-f989d6030b0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:24:01.788807 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f28b9d-eb0e-489a-a913-f989d6030b0b admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:24:01.789219 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f28b9d-eb0e-489a-a913-f989d6030b0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:24:01.789574 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f28b9d-eb0e-489a-a913-f989d6030b0b admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:24:01.790362 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f28b9d-eb0e-489a-a913-f989d6030b0b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:24:01.792477 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f28b9d-eb0e-489a-a913-f989d6030b0b admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:24:01.813020 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:24:01.823011 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1402/2833] 10.222.0.99 () {60 vars in 1152 bytes} [Thu Jul 14 16:24:01 2022] POST /identity/v3/projects => generated 389 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 16:24:01.827199 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbe5414-012a-4447-840a-3cba42ab6d8f admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:24:01.838640 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbe5414-012a-4447-840a-3cba42ab6d8f admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:24:01.876472 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:24:01.905001 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba9f90f-a812-4a39-8cdb-9984059ee2a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:24:01.905682 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba9f90f-a812-4a39-8cdb-9984059ee2a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:24:01.906001 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba9f90f-a812-4a39-8cdb-9984059ee2a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:24:01.906286 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba9f90f-a812-4a39-8cdb-9984059ee2a8 admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:24:01.906991 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba9f90f-a812-4a39-8cdb-9984059ee2a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:24:01.908944 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba9f90f-a812-4a39-8cdb-9984059ee2a8 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:24:01.929146 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1432/2834] 10.222.0.99 () {60 vars in 1143 bytes} [Thu Jul 14 16:24:01 2022] POST /identity/v3/users => generated 295 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 16:24:01.933647 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f28b9d-eb0e-489a-a913-f989d6030b0b admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:24:01.944573 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f28b9d-eb0e-489a-a913-f989d6030b0b admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:24:01.949988 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b429305b-c43b-478d-895c-2c3c01f8f871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:24:01.950645 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b429305b-c43b-478d-895c-2c3c01f8f871 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:24:01.950932 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b429305b-c43b-478d-895c-2c3c01f8f871 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:24:01.951215 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b429305b-c43b-478d-895c-2c3c01f8f871 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:24:01.951983 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b429305b-c43b-478d-895c-2c3c01f8f871 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:24:01.953575 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b429305b-c43b-478d-895c-2c3c01f8f871 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:24:01.964132 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1403/2835] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 16:24:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:24:01.968032 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba9f90f-a812-4a39-8cdb-9984059ee2a8 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:24:01.978227 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba9f90f-a812-4a39-8cdb-9984059ee2a8 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:24:01.982762 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa14fb65-7bd2-419a-9bce-db7ce1e1895c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:24:01.983423 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa14fb65-7bd2-419a-9bce-db7ce1e1895c admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:24:01.983789 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa14fb65-7bd2-419a-9bce-db7ce1e1895c admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:24:01.984149 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa14fb65-7bd2-419a-9bce-db7ce1e1895c admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:24:01.984908 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa14fb65-7bd2-419a-9bce-db7ce1e1895c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:24:01.986537 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa14fb65-7bd2-419a-9bce-db7ce1e1895c admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:24:01.997576 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1433/2836] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 16:24:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 16:24:02.001764 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b429305b-c43b-478d-895c-2c3c01f8f871 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:24:02.013125 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b429305b-c43b-478d-895c-2c3c01f8f871 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:24:02.017660 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261c6fc6-b86e-4abe-9c6f-c607fd3a25ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:24:02.018545 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261c6fc6-b86e-4abe-9c6f-c607fd3a25ef admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:24:02.018832 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261c6fc6-b86e-4abe-9c6f-c607fd3a25ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:24:02.019116 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261c6fc6-b86e-4abe-9c6f-c607fd3a25ef admin admin] PATH_INFO: `/v3/projects/e050a55b04aa4791aabf6f72ff84b71a/users/354ab2c8531d4544b7728605ca321eed/roles/b8c98fddc0fd4436b577fd1d3361bc38` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:24:02.034390 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261c6fc6-b86e-4abe-9c6f-c607fd3a25ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=e050a55b04aa4791aabf6f72ff84b71a, user_id=354ab2c8531d4544b7728605ca321eed, role_id=b8c98fddc0fd4436b577fd1d3361bc38)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:24:02.037052 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261c6fc6-b86e-4abe-9c6f-c607fd3a25ef admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:24:02.049823 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1404/2837] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 16:24:02 2022] PUT /identity/v3/projects/e050a55b04aa4791aabf6f72ff84b71a/users/354ab2c8531d4544b7728605ca321eed/roles/b8c98fddc0fd4436b577fd1d3361bc38 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:24:02.062298 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159911f2-e0db-465f-974d-6962e1d6c6dd None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:24:02.062597 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159911f2-e0db-465f-974d-6962e1d6c6dd None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:24:02.062884 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159911f2-e0db-465f-974d-6962e1d6c6dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:24:02.089171 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-159911f2-e0db-465f-974d-6962e1d6c6dd None None] MFA Rules not processed for user `354ab2c8531d4544b7728605ca321eed`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:24:02.090305 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-159911f2-e0db-465f-974d-6962e1d6c6dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:24:02.155228 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1434/2838] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 16:24:02 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:24:02.159934 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261c6fc6-b86e-4abe-9c6f-c607fd3a25ef admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:24:02.222539 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261c6fc6-b86e-4abe-9c6f-c607fd3a25ef admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:24:02.228275 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11a6e2f-ad3d-48e4-9019-4cb7e92ad4ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:24:02.228935 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11a6e2f-ad3d-48e4-9019-4cb7e92ad4ce admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:24:02.229223 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11a6e2f-ad3d-48e4-9019-4cb7e92ad4ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:24:02.229505 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11a6e2f-ad3d-48e4-9019-4cb7e92ad4ce admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:24:02.230189 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11a6e2f-ad3d-48e4-9019-4cb7e92ad4ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:24:02.232134 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11a6e2f-ad3d-48e4-9019-4cb7e92ad4ce admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:24:02.258655 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1405/2839] 10.222.0.99 () {60 vars in 1152 bytes} [Thu Jul 14 16:24:02 2022] POST /identity/v3/projects => generated 387 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 16:24:02.262811 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159911f2-e0db-465f-974d-6962e1d6c6dd None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:24:02.273786 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159911f2-e0db-465f-974d-6962e1d6c6dd None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:24:02.331731 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89822615-5484-4c65-9e6c-3e55d03e4e8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:24:02.332406 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89822615-5484-4c65-9e6c-3e55d03e4e8e admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:24:02.332693 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89822615-5484-4c65-9e6c-3e55d03e4e8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:24:02.332973 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89822615-5484-4c65-9e6c-3e55d03e4e8e admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:24:02.333675 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89822615-5484-4c65-9e6c-3e55d03e4e8e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:24:02.339103 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89822615-5484-4c65-9e6c-3e55d03e4e8e admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:24:02.360189 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1435/2840] 10.222.0.99 () {60 vars in 1143 bytes} [Thu Jul 14 16:24:02 2022] POST /identity/v3/users => generated 300 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 16:24:02.364151 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11a6e2f-ad3d-48e4-9019-4cb7e92ad4ce admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:24:02.375381 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11a6e2f-ad3d-48e4-9019-4cb7e92ad4ce admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:24:02.380503 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bbec9a-3b50-411e-8e1c-961b10ccf836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:24:02.381203 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bbec9a-3b50-411e-8e1c-961b10ccf836 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:24:02.382035 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bbec9a-3b50-411e-8e1c-961b10ccf836 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:24:02.382318 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bbec9a-3b50-411e-8e1c-961b10ccf836 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:24:02.383050 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bbec9a-3b50-411e-8e1c-961b10ccf836 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:24:02.384711 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bbec9a-3b50-411e-8e1c-961b10ccf836 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:24:02.396524 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1406/2841] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 16:24:02 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 16:24:02.401704 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89822615-5484-4c65-9e6c-3e55d03e4e8e admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:24:02.411973 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89822615-5484-4c65-9e6c-3e55d03e4e8e admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:24:02.416917 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af8df0d-ed54-41e1-b9b3-ad260f0c94fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:24:02.417796 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af8df0d-ed54-41e1-b9b3-ad260f0c94fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:24:02.418087 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af8df0d-ed54-41e1-b9b3-ad260f0c94fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:24:02.418371 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af8df0d-ed54-41e1-b9b3-ad260f0c94fe admin admin] PATH_INFO: `/v3/projects/c3448e6659014afaa63110c04fec5a15/users/a82ea01b950744e2a72ac8db6a848acb/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:24:02.434071 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af8df0d-ed54-41e1-b9b3-ad260f0c94fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3448e6659014afaa63110c04fec5a15, user_id=a82ea01b950744e2a72ac8db6a848acb, role_id=d0f6ba1314f14d9f9c816bdb5d0c3aac)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:24:02.436895 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af8df0d-ed54-41e1-b9b3-ad260f0c94fe admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:24:02.450435 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1436/2842] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 16:24:02 2022] PUT /identity/v3/projects/c3448e6659014afaa63110c04fec5a15/users/a82ea01b950744e2a72ac8db6a848acb/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:24:02.454360 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bbec9a-3b50-411e-8e1c-961b10ccf836 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:24:02.464517 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bbec9a-3b50-411e-8e1c-961b10ccf836 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:24:02.522434 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65051e2-898c-4219-913f-4fbbf54a0614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:24:02.523108 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65051e2-898c-4219-913f-4fbbf54a0614 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:24:02.523392 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65051e2-898c-4219-913f-4fbbf54a0614 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:24:02.523700 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65051e2-898c-4219-913f-4fbbf54a0614 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:24:02.524427 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65051e2-898c-4219-913f-4fbbf54a0614 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:24:02.526024 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65051e2-898c-4219-913f-4fbbf54a0614 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:24:02.536148 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1407/2843] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 16:24:02 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:24:02.540314 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af8df0d-ed54-41e1-b9b3-ad260f0c94fe admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:24:02.551000 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af8df0d-ed54-41e1-b9b3-ad260f0c94fe admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:24:02.556263 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16be8522-9b44-4ad4-8de6-585c919a20f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:24:02.557440 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16be8522-9b44-4ad4-8de6-585c919a20f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:24:02.557720 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16be8522-9b44-4ad4-8de6-585c919a20f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:24:02.558019 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16be8522-9b44-4ad4-8de6-585c919a20f1 admin admin] PATH_INFO: `/v3/projects/c3448e6659014afaa63110c04fec5a15/users/a82ea01b950744e2a72ac8db6a848acb/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:24:02.560018 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16be8522-9b44-4ad4-8de6-585c919a20f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3448e6659014afaa63110c04fec5a15, user_id=a82ea01b950744e2a72ac8db6a848acb, role_id=d0f6ba1314f14d9f9c816bdb5d0c3aac)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:24:02.562655 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16be8522-9b44-4ad4-8de6-585c919a20f1 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:24:02.566086 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:24:02.577531 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1437/2844] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 16:24:02 2022] PUT /identity/v3/projects/c3448e6659014afaa63110c04fec5a15/users/a82ea01b950744e2a72ac8db6a848acb/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:24:02.588740 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29835dc-e518-48f5-b03d-269859f69c85 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:24:02.589270 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29835dc-e518-48f5-b03d-269859f69c85 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:24:02.589581 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29835dc-e518-48f5-b03d-269859f69c85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:24:02.618218 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-c29835dc-e518-48f5-b03d-269859f69c85 None None] MFA Rules not processed for user `a82ea01b950744e2a72ac8db6a848acb`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:24:02.619358 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c29835dc-e518-48f5-b03d-269859f69c85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:24:02.640830 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:24:02.702771 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1408/2845] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 16:24:02 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:24:02.714341 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16be8522-9b44-4ad4-8de6-585c919a20f1 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:24:02.716141 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-16be8522-9b44-4ad4-8de6-585c919a20f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:24:02.802400 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16be8522-9b44-4ad4-8de6-585c919a20f1 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:24:02.809722 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12a4943-110f-4d94-85bc-4822b9b697f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3e1a2b83f3d4d88ba5084dfaffd22f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:24:02.810832 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12a4943-110f-4d94-85bc-4822b9b697f2 service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:24:02.811334 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12a4943-110f-4d94-85bc-4822b9b697f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:24:02.811908 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12a4943-110f-4d94-85bc-4822b9b697f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:24:02.824490 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12a4943-110f-4d94-85bc-4822b9b697f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:24:02.825963 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12a4943-110f-4d94-85bc-4822b9b697f2 service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:24:02.836640 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1438/2846] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:24:02 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:24:02.972999 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29835dc-e518-48f5-b03d-269859f69c85 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:24:03.040182 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29835dc-e518-48f5-b03d-269859f69c85 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:24:03.044770 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3010a708-e6b1-42b7-8c33-828e03d1c848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:24:03.045654 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3010a708-e6b1-42b7-8c33-828e03d1c848 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:24:03.045943 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3010a708-e6b1-42b7-8c33-828e03d1c848 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:24:03.046229 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3010a708-e6b1-42b7-8c33-828e03d1c848 admin admin] PATH_INFO: `/v3/users/354ab2c8531d4544b7728605ca321eed` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:24:03.047442 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3010a708-e6b1-42b7-8c33-828e03d1c848 admin admin] RBAC: Authorizing `identity:delete_user(user_id=354ab2c8531d4544b7728605ca321eed)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:24:03.049639 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3010a708-e6b1-42b7-8c33-828e03d1c848 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:24:03.098464 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-3010a708-e6b1-42b7-8c33-828e03d1c848 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '354ab2c8531d4544b7728605ca321eed'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:24:03.107363 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-3010a708-e6b1-42b7-8c33-828e03d1c848 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '354ab2c8531d4544b7728605ca321eed'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:24:03.120422 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-3010a708-e6b1-42b7-8c33-828e03d1c848 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '354ab2c8531d4544b7728605ca321eed'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:24:03.121051 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-3010a708-e6b1-42b7-8c33-828e03d1c848 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '354ab2c8531d4544b7728605ca321eed'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:24:03.121654 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-3010a708-e6b1-42b7-8c33-828e03d1c848 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '354ab2c8531d4544b7728605ca321eed'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:24:03.136248 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1409/2847] 10.222.0.99 () {58 vars in 1223 bytes} [Thu Jul 14 16:24:02 2022] DELETE /identity/v3/users/354ab2c8531d4544b7728605ca321eed => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 16:24:03.142130 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece34abf-999d-4048-b9b1-e0214f0f1989 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:24:03.142665 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece34abf-999d-4048-b9b1-e0214f0f1989 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:24:03.142789 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece34abf-999d-4048-b9b1-e0214f0f1989 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:24:03.164462 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-ece34abf-999d-4048-b9b1-e0214f0f1989 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:24:03.165710 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-ece34abf-999d-4048-b9b1-e0214f0f1989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:24:03.229009 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1439/2848] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 16:24:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:24:03.238979 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3010a708-e6b1-42b7-8c33-828e03d1c848 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:24:03.240459 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-3010a708-e6b1-42b7-8c33-828e03d1c848 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:24:03.284301 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:24:03.302673 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3010a708-e6b1-42b7-8c33-828e03d1c848 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:24:03.308997 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982c7277-5c9b-4d1d-b578-42e755c36934 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:24:03.309700 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982c7277-5c9b-4d1d-b578-42e755c36934 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:24:03.309995 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982c7277-5c9b-4d1d-b578-42e755c36934 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:24:03.310304 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982c7277-5c9b-4d1d-b578-42e755c36934 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:24:03.321325 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982c7277-5c9b-4d1d-b578-42e755c36934 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:24:03.322883 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982c7277-5c9b-4d1d-b578-42e755c36934 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:24:03.333905 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1410/2849] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:24:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:24:03.640412 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece34abf-999d-4048-b9b1-e0214f0f1989 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:24:03.641866 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-ece34abf-999d-4048-b9b1-e0214f0f1989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:24:03.655351 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece34abf-999d-4048-b9b1-e0214f0f1989 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:24:03.664324 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa86cbb-8ec6-49bd-9a51-ca855b16c7a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:24:03.665224 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa86cbb-8ec6-49bd-9a51-ca855b16c7a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:24:03.665513 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa86cbb-8ec6-49bd-9a51-ca855b16c7a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:24:03.665878 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa86cbb-8ec6-49bd-9a51-ca855b16c7a7 admin admin] PATH_INFO: `/v3/projects/e050a55b04aa4791aabf6f72ff84b71a` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:24:03.667065 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa86cbb-8ec6-49bd-9a51-ca855b16c7a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e050a55b04aa4791aabf6f72ff84b71a)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:24:03.669043 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa86cbb-8ec6-49bd-9a51-ca855b16c7a7 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:24:03.725431 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-2aa86cbb-8ec6-49bd-9a51-ca855b16c7a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e050a55b04aa4791aabf6f72ff84b71a', 'request_id': 'req-2aa86cbb-8ec6-49bd-9a51-ca855b16c7a7'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:24:03.732461 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-2aa86cbb-8ec6-49bd-9a51-ca855b16c7a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e050a55b04aa4791aabf6f72ff84b71a', 'request_id': 'req-2aa86cbb-8ec6-49bd-9a51-ca855b16c7a7'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:24:03.738114 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-2aa86cbb-8ec6-49bd-9a51-ca855b16c7a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e050a55b04aa4791aabf6f72ff84b71a', 'request_id': 'req-2aa86cbb-8ec6-49bd-9a51-ca855b16c7a7'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:24:03.751443 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-2aa86cbb-8ec6-49bd-9a51-ca855b16c7a7 admin admin] The token cache is being invalidate because project e050a55b04aa4791aabf6f72ff84b71a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 16:24:03.751758 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-2aa86cbb-8ec6-49bd-9a51-ca855b16c7a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:24:03.752392 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-2aa86cbb-8ec6-49bd-9a51-ca855b16c7a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:24:03.753938 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1440/2850] 10.222.0.99 () {58 vars in 1232 bytes} [Thu Jul 14 16:24:03 2022] DELETE /identity/v3/projects/e050a55b04aa4791aabf6f72ff84b71a => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:24:03.758841 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982c7277-5c9b-4d1d-b578-42e755c36934 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:24:03.768716 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982c7277-5c9b-4d1d-b578-42e755c36934 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:24:03.827005 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ab361e-6677-4ff1-9409-ba27b2d37eae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:24:03.827902 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ab361e-6677-4ff1-9409-ba27b2d37eae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:24:03.828185 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ab361e-6677-4ff1-9409-ba27b2d37eae admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:24:03.828491 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ab361e-6677-4ff1-9409-ba27b2d37eae admin admin] PATH_INFO: `/v3/users/a82ea01b950744e2a72ac8db6a848acb` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:24:03.829684 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ab361e-6677-4ff1-9409-ba27b2d37eae admin admin] RBAC: Authorizing `identity:delete_user(user_id=a82ea01b950744e2a72ac8db6a848acb)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:24:03.832045 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ab361e-6677-4ff1-9409-ba27b2d37eae admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:24:03.880210 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e2ab361e-6677-4ff1-9409-ba27b2d37eae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a82ea01b950744e2a72ac8db6a848acb'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:24:03.896421 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e2ab361e-6677-4ff1-9409-ba27b2d37eae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a82ea01b950744e2a72ac8db6a848acb'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:24:03.909839 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e2ab361e-6677-4ff1-9409-ba27b2d37eae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a82ea01b950744e2a72ac8db6a848acb'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:24:03.911088 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e2ab361e-6677-4ff1-9409-ba27b2d37eae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a82ea01b950744e2a72ac8db6a848acb'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:24:03.911740 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e2ab361e-6677-4ff1-9409-ba27b2d37eae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a82ea01b950744e2a72ac8db6a848acb'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:24:03.924585 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1411/2851] 10.222.0.99 () {58 vars in 1223 bytes} [Thu Jul 14 16:24:03 2022] DELETE /identity/v3/users/a82ea01b950744e2a72ac8db6a848acb => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 16:24:04.221070 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa86cbb-8ec6-49bd-9a51-ca855b16c7a7 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:24:04.222558 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-2aa86cbb-8ec6-49bd-9a51-ca855b16c7a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:24:04.238017 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa86cbb-8ec6-49bd-9a51-ca855b16c7a7 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:24:04.249789 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:24:04.305256 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3eb7171-c7a9-44cf-b598-1d602b66365c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:24:04.306184 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eb7171-c7a9-44cf-b598-1d602b66365c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:24:04.306734 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eb7171-c7a9-44cf-b598-1d602b66365c admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:24:04.307019 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eb7171-c7a9-44cf-b598-1d602b66365c admin admin] PATH_INFO: `/v3/projects/c3448e6659014afaa63110c04fec5a15` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:24:04.308382 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3eb7171-c7a9-44cf-b598-1d602b66365c admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3448e6659014afaa63110c04fec5a15)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:24:04.310362 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3eb7171-c7a9-44cf-b598-1d602b66365c admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:24:04.358793 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-b3eb7171-c7a9-44cf-b598-1d602b66365c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3448e6659014afaa63110c04fec5a15', 'request_id': 'req-b3eb7171-c7a9-44cf-b598-1d602b66365c'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:24:04.365073 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-b3eb7171-c7a9-44cf-b598-1d602b66365c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3448e6659014afaa63110c04fec5a15', 'request_id': 'req-b3eb7171-c7a9-44cf-b598-1d602b66365c'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:24:04.371929 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-b3eb7171-c7a9-44cf-b598-1d602b66365c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3448e6659014afaa63110c04fec5a15', 'request_id': 'req-b3eb7171-c7a9-44cf-b598-1d602b66365c'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:24:04.385391 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-b3eb7171-c7a9-44cf-b598-1d602b66365c admin admin] The token cache is being invalidate because project c3448e6659014afaa63110c04fec5a15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 16:24:04.385685 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-b3eb7171-c7a9-44cf-b598-1d602b66365c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:24:04.386317 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-b3eb7171-c7a9-44cf-b598-1d602b66365c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:24:04.387932 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1441/2852] 10.222.0.99 () {58 vars in 1232 bytes} [Thu Jul 14 16:24:04 2022] DELETE /identity/v3/projects/c3448e6659014afaa63110c04fec5a15 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 16:24:04.392939 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1b56ee-e5c8-4afa-a5bb-6edc9ea4e600 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:24:04.393423 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1b56ee-e5c8-4afa-a5bb-6edc9ea4e600 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:24:04.393525 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1b56ee-e5c8-4afa-a5bb-6edc9ea4e600 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:24:04.412003 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-2b1b56ee-e5c8-4afa-a5bb-6edc9ea4e600 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:24:04.414276 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-2b1b56ee-e5c8-4afa-a5bb-6edc9ea4e600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:24:04.470546 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:24:04.482104 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1412/2853] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 16:24:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:24:04.490491 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d163f25-002f-406b-870d-ee6a7b57320d None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:24:04.491435 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d163f25-002f-406b-870d-ee6a7b57320d None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:24:04.491958 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d163f25-002f-406b-870d-ee6a7b57320d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:24:04.510916 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-1d163f25-002f-406b-870d-ee6a7b57320d None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:24:04.512098 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-1d163f25-002f-406b-870d-ee6a7b57320d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:24:04.520109 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1442/2854] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 16:24:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:24:04.524635 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1b56ee-e5c8-4afa-a5bb-6edc9ea4e600 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:24:04.534517 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1b56ee-e5c8-4afa-a5bb-6edc9ea4e600 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:24:04.539716 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5b4be8-c31b-4d96-928f-a52a4ae75646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:24:04.540371 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5b4be8-c31b-4d96-928f-a52a4ae75646 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:24:04.540656 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5b4be8-c31b-4d96-928f-a52a4ae75646 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:24:04.540932 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5b4be8-c31b-4d96-928f-a52a4ae75646 admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:24:04.541656 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5b4be8-c31b-4d96-928f-a52a4ae75646 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 16:24:04.541962 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5b4be8-c31b-4d96-928f-a52a4ae75646 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:24:04.543574 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5b4be8-c31b-4d96-928f-a52a4ae75646 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:24:04.557132 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1413/2855] 10.222.0.99 () {58 vars in 1152 bytes} [Thu Jul 14 16:24:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 16:24:04.561432 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d163f25-002f-406b-870d-ee6a7b57320d None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:24:04.571174 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d163f25-002f-406b-870d-ee6a7b57320d None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:24:04.579630 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57fd125-2d5b-4fde-8569-018eac1a5889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:24:04.580300 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57fd125-2d5b-4fde-8569-018eac1a5889 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:24:04.580591 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57fd125-2d5b-4fde-8569-018eac1a5889 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:24:04.580874 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57fd125-2d5b-4fde-8569-018eac1a5889 admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:24:04.581584 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57fd125-2d5b-4fde-8569-018eac1a5889 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:24:04.583509 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57fd125-2d5b-4fde-8569-018eac1a5889 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:24:04.610142 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1443/2856] 10.222.0.99 () {60 vars in 1152 bytes} [Thu Jul 14 16:24:04 2022] POST /identity/v3/projects => generated 375 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 16:24:04.614282 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5b4be8-c31b-4d96-928f-a52a4ae75646 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:24:04.624974 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5b4be8-c31b-4d96-928f-a52a4ae75646 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:24:04.684204 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56fe0cb-de26-4f32-9cfd-7aafb53dc895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:24:04.684879 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56fe0cb-de26-4f32-9cfd-7aafb53dc895 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:24:04.685164 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56fe0cb-de26-4f32-9cfd-7aafb53dc895 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:24:04.685440 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56fe0cb-de26-4f32-9cfd-7aafb53dc895 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:24:04.686133 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56fe0cb-de26-4f32-9cfd-7aafb53dc895 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:24:04.688083 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56fe0cb-de26-4f32-9cfd-7aafb53dc895 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:24:04.707107 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1414/2857] 10.222.0.99 () {60 vars in 1143 bytes} [Thu Jul 14 16:24:04 2022] POST /identity/v3/users => generated 288 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 16:24:04.712445 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57fd125-2d5b-4fde-8569-018eac1a5889 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:24:04.722807 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57fd125-2d5b-4fde-8569-018eac1a5889 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:24:04.728317 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b425aa1e-6264-472b-aa6a-33583ef76c6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:24:04.729011 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b425aa1e-6264-472b-aa6a-33583ef76c6a admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:24:04.729280 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b425aa1e-6264-472b-aa6a-33583ef76c6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:24:04.729569 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b425aa1e-6264-472b-aa6a-33583ef76c6a admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:24:04.730323 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b425aa1e-6264-472b-aa6a-33583ef76c6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:24:04.732012 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b425aa1e-6264-472b-aa6a-33583ef76c6a admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:24:04.743975 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1444/2858] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 16:24:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:24:04.748595 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56fe0cb-de26-4f32-9cfd-7aafb53dc895 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:24:04.759382 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56fe0cb-de26-4f32-9cfd-7aafb53dc895 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:24:04.764656 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a35c27e-67e0-4827-be59-e14604264f2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:24:04.765276 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a35c27e-67e0-4827-be59-e14604264f2c admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:24:04.765572 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a35c27e-67e0-4827-be59-e14604264f2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:24:04.765859 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a35c27e-67e0-4827-be59-e14604264f2c admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:24:04.766602 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a35c27e-67e0-4827-be59-e14604264f2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:24:04.768263 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a35c27e-67e0-4827-be59-e14604264f2c admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:24:04.781888 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1415/2859] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 16:24:04 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:24:04.786154 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b425aa1e-6264-472b-aa6a-33583ef76c6a admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:24:04.798084 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b425aa1e-6264-472b-aa6a-33583ef76c6a admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:24:04.802821 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690120ba-8e78-48a3-b610-eb11d06c7d5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:24:04.803795 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690120ba-8e78-48a3-b610-eb11d06c7d5d admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:24:04.804110 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690120ba-8e78-48a3-b610-eb11d06c7d5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:24:04.804398 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690120ba-8e78-48a3-b610-eb11d06c7d5d admin admin] PATH_INFO: `/v3/projects/588fcdcf9d34452889ebd1184ea6c62a/users/d3eb51ff6bdf42708dab1cb3e0e861a2/roles/b8c98fddc0fd4436b577fd1d3361bc38` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:24:04.820008 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690120ba-8e78-48a3-b610-eb11d06c7d5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=588fcdcf9d34452889ebd1184ea6c62a, user_id=d3eb51ff6bdf42708dab1cb3e0e861a2, role_id=b8c98fddc0fd4436b577fd1d3361bc38)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:24:04.822665 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690120ba-8e78-48a3-b610-eb11d06c7d5d admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:24:04.837892 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1445/2860] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 16:24:04 2022] PUT /identity/v3/projects/588fcdcf9d34452889ebd1184ea6c62a/users/d3eb51ff6bdf42708dab1cb3e0e861a2/roles/b8c98fddc0fd4436b577fd1d3361bc38 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:24:04.848238 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0f5020-1d0a-4211-8078-e3252b7a1814 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:24:04.848538 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0f5020-1d0a-4211-8078-e3252b7a1814 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:24:04.848826 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0f5020-1d0a-4211-8078-e3252b7a1814 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:24:04.875616 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-7f0f5020-1d0a-4211-8078-e3252b7a1814 None None] MFA Rules not processed for user `d3eb51ff6bdf42708dab1cb3e0e861a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:24:04.876859 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-7f0f5020-1d0a-4211-8078-e3252b7a1814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:24:04.960871 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1416/2861] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 16:24:04 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:24:04.965380 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690120ba-8e78-48a3-b610-eb11d06c7d5d admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:24:05.014592 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:24:05.037106 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690120ba-8e78-48a3-b610-eb11d06c7d5d admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:24:05.042664 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e8864b-a335-4a2c-a155-915f0c52c9af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:24:05.043549 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e8864b-a335-4a2c-a155-915f0c52c9af admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:24:05.043713 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e8864b-a335-4a2c-a155-915f0c52c9af admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:24:05.043950 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e8864b-a335-4a2c-a155-915f0c52c9af admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:24:05.044650 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e8864b-a335-4a2c-a155-915f0c52c9af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:24:05.046567 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e8864b-a335-4a2c-a155-915f0c52c9af admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:24:05.074120 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1446/2862] 10.222.0.99 () {60 vars in 1152 bytes} [Thu Jul 14 16:24:04 2022] POST /identity/v3/projects => generated 375 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 16:24:05.078256 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0f5020-1d0a-4211-8078-e3252b7a1814 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:24:05.090562 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0f5020-1d0a-4211-8078-e3252b7a1814 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:24:05.140694 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:24:05.150189 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d2c489-97e4-49e5-9b2f-e50f89d7cd0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:24:05.150854 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d2c489-97e4-49e5-9b2f-e50f89d7cd0a admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:24:05.151191 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d2c489-97e4-49e5-9b2f-e50f89d7cd0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:24:05.151450 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d2c489-97e4-49e5-9b2f-e50f89d7cd0a admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:24:05.152252 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d2c489-97e4-49e5-9b2f-e50f89d7cd0a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:24:05.154180 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d2c489-97e4-49e5-9b2f-e50f89d7cd0a admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:24:05.175379 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1417/2863] 10.222.0.99 () {60 vars in 1143 bytes} [Thu Jul 14 16:24:05 2022] POST /identity/v3/users => generated 288 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 16:24:05.179831 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e8864b-a335-4a2c-a155-915f0c52c9af admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:24:05.190982 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e8864b-a335-4a2c-a155-915f0c52c9af admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:24:05.196328 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc53816-9087-4af0-9359-3a94696dc894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:24:05.197003 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc53816-9087-4af0-9359-3a94696dc894 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:24:05.197297 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc53816-9087-4af0-9359-3a94696dc894 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:24:05.198077 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc53816-9087-4af0-9359-3a94696dc894 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:24:05.198441 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc53816-9087-4af0-9359-3a94696dc894 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:24:05.200098 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc53816-9087-4af0-9359-3a94696dc894 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:24:05.212017 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1447/2864] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 16:24:05 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 16:24:05.216096 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d2c489-97e4-49e5-9b2f-e50f89d7cd0a admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:24:05.226622 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d2c489-97e4-49e5-9b2f-e50f89d7cd0a admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:24:05.231296 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc1168e-1e18-4389-af32-d570f64fe169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:24:05.232068 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc1168e-1e18-4389-af32-d570f64fe169 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:24:05.232356 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc1168e-1e18-4389-af32-d570f64fe169 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:24:05.232641 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc1168e-1e18-4389-af32-d570f64fe169 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:24:05.233382 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc1168e-1e18-4389-af32-d570f64fe169 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:24:05.234989 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc1168e-1e18-4389-af32-d570f64fe169 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:24:05.245832 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1418/2865] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 16:24:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:24:05.250239 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc53816-9087-4af0-9359-3a94696dc894 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:24:05.261531 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc53816-9087-4af0-9359-3a94696dc894 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:24:05.266397 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20a6c93-e812-48a1-aabd-8e0815a10e95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:24:05.267298 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20a6c93-e812-48a1-aabd-8e0815a10e95 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:24:05.267588 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20a6c93-e812-48a1-aabd-8e0815a10e95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:24:05.267898 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20a6c93-e812-48a1-aabd-8e0815a10e95 admin admin] PATH_INFO: `/v3/projects/b8b0fb6310ef4f4092511cceb2b7b2ec/users/55545650385f4b98a8f25c0e0949062c/roles/b8c98fddc0fd4436b577fd1d3361bc38` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:24:05.284196 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20a6c93-e812-48a1-aabd-8e0815a10e95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8b0fb6310ef4f4092511cceb2b7b2ec, user_id=55545650385f4b98a8f25c0e0949062c, role_id=b8c98fddc0fd4436b577fd1d3361bc38)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:24:05.286837 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20a6c93-e812-48a1-aabd-8e0815a10e95 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:24:05.301448 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1448/2866] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 16:24:05 2022] PUT /identity/v3/projects/b8b0fb6310ef4f4092511cceb2b7b2ec/users/55545650385f4b98a8f25c0e0949062c/roles/b8c98fddc0fd4436b577fd1d3361bc38 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:24:05.312636 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df07ecae-66af-475a-b7f9-ebdbbe4d1f0e None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:24:05.312931 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df07ecae-66af-475a-b7f9-ebdbbe4d1f0e None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:24:05.313216 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df07ecae-66af-475a-b7f9-ebdbbe4d1f0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:24:05.341482 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-df07ecae-66af-475a-b7f9-ebdbbe4d1f0e None None] MFA Rules not processed for user `55545650385f4b98a8f25c0e0949062c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:24:05.342709 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-df07ecae-66af-475a-b7f9-ebdbbe4d1f0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:24:05.419280 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1419/2867] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 16:24:05 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:24:05.423865 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20a6c93-e812-48a1-aabd-8e0815a10e95 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:24:05.488445 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20a6c93-e812-48a1-aabd-8e0815a10e95 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:24:05.494342 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768997ae-3793-419d-b3fc-c5864b238d2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:24:05.495022 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768997ae-3793-419d-b3fc-c5864b238d2d admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:24:05.495326 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768997ae-3793-419d-b3fc-c5864b238d2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:24:05.495788 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768997ae-3793-419d-b3fc-c5864b238d2d admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:24:05.496653 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768997ae-3793-419d-b3fc-c5864b238d2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:24:05.500339 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768997ae-3793-419d-b3fc-c5864b238d2d admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:24:05.534478 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1449/2868] 10.222.0.99 () {60 vars in 1152 bytes} [Thu Jul 14 16:24:05 2022] POST /identity/v3/projects => generated 377 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 16:24:05.538589 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df07ecae-66af-475a-b7f9-ebdbbe4d1f0e None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:24:05.549911 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df07ecae-66af-475a-b7f9-ebdbbe4d1f0e None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:24:05.611918 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae33a23-98d4-4501-b5ef-3d41ba5916e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:24:05.612519 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae33a23-98d4-4501-b5ef-3d41ba5916e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:24:05.612813 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae33a23-98d4-4501-b5ef-3d41ba5916e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:24:05.613088 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae33a23-98d4-4501-b5ef-3d41ba5916e3 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:24:05.613782 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae33a23-98d4-4501-b5ef-3d41ba5916e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:24:05.615729 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae33a23-98d4-4501-b5ef-3d41ba5916e3 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:24:05.636874 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1420/2869] 10.222.0.99 () {60 vars in 1143 bytes} [Thu Jul 14 16:24:05 2022] POST /identity/v3/users => generated 295 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 16:24:05.640980 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768997ae-3793-419d-b3fc-c5864b238d2d admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:24:05.652097 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768997ae-3793-419d-b3fc-c5864b238d2d admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:24:05.656751 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b7b52a-2dd2-4d6c-b835-316c820509d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:24:05.657411 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b7b52a-2dd2-4d6c-b835-316c820509d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:24:05.657706 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b7b52a-2dd2-4d6c-b835-316c820509d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:24:05.657988 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b7b52a-2dd2-4d6c-b835-316c820509d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:24:05.658751 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b7b52a-2dd2-4d6c-b835-316c820509d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:24:05.660397 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b7b52a-2dd2-4d6c-b835-316c820509d3 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:24:05.671712 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1450/2870] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 16:24:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 16:24:05.675867 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae33a23-98d4-4501-b5ef-3d41ba5916e3 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:24:05.686747 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae33a23-98d4-4501-b5ef-3d41ba5916e3 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:24:05.691685 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26091bb4-5682-4ff8-98a5-cea6bc8e985e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:24:05.692599 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26091bb4-5682-4ff8-98a5-cea6bc8e985e admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:24:05.692886 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26091bb4-5682-4ff8-98a5-cea6bc8e985e admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:24:05.693198 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26091bb4-5682-4ff8-98a5-cea6bc8e985e admin admin] PATH_INFO: `/v3/projects/17a1648f4a8d4babaef43532d9f67c49/users/59729b399bed476ea439f1ef28893455/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:24:05.708763 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26091bb4-5682-4ff8-98a5-cea6bc8e985e admin admin] RBAC: Authorizing `identity:create_grant(project_id=17a1648f4a8d4babaef43532d9f67c49, user_id=59729b399bed476ea439f1ef28893455, role_id=d0f6ba1314f14d9f9c816bdb5d0c3aac)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:24:05.712085 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26091bb4-5682-4ff8-98a5-cea6bc8e985e admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:24:05.728272 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1421/2871] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 16:24:05 2022] PUT /identity/v3/projects/17a1648f4a8d4babaef43532d9f67c49/users/59729b399bed476ea439f1ef28893455/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:24:05.732545 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b7b52a-2dd2-4d6c-b835-316c820509d3 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:24:05.746491 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b7b52a-2dd2-4d6c-b835-316c820509d3 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:24:05.809167 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387e7a93-1412-448d-8bcf-ce643a8acdbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:24:05.809842 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387e7a93-1412-448d-8bcf-ce643a8acdbc admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:24:05.810132 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387e7a93-1412-448d-8bcf-ce643a8acdbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:24:05.810413 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387e7a93-1412-448d-8bcf-ce643a8acdbc admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:24:05.811145 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387e7a93-1412-448d-8bcf-ce643a8acdbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:24:05.812785 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387e7a93-1412-448d-8bcf-ce643a8acdbc admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:24:05.818082 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:24:05.824527 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1451/2872] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 16:24:05 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:24:05.828668 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26091bb4-5682-4ff8-98a5-cea6bc8e985e admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:24:05.840168 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26091bb4-5682-4ff8-98a5-cea6bc8e985e admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:24:05.844757 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e54c25-77ad-4ec4-ab91-4d59bb7f02d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:24:05.845646 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e54c25-77ad-4ec4-ab91-4d59bb7f02d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:24:05.845931 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e54c25-77ad-4ec4-ab91-4d59bb7f02d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:24:05.846208 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e54c25-77ad-4ec4-ab91-4d59bb7f02d1 admin admin] PATH_INFO: `/v3/projects/17a1648f4a8d4babaef43532d9f67c49/users/59729b399bed476ea439f1ef28893455/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:24:05.848217 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e54c25-77ad-4ec4-ab91-4d59bb7f02d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17a1648f4a8d4babaef43532d9f67c49, user_id=59729b399bed476ea439f1ef28893455, role_id=d0f6ba1314f14d9f9c816bdb5d0c3aac)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:24:05.850854 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e54c25-77ad-4ec4-ab91-4d59bb7f02d1 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:24:05.861990 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1422/2873] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 16:24:05 2022] PUT /identity/v3/projects/17a1648f4a8d4babaef43532d9f67c49/users/59729b399bed476ea439f1ef28893455/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 16:24:05.873819 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb2765e-6432-4790-ab72-cde24bc347aa None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:24:05.874111 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb2765e-6432-4790-ab72-cde24bc347aa None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:24:05.874390 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb2765e-6432-4790-ab72-cde24bc347aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:24:05.903835 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-ecb2765e-6432-4790-ab72-cde24bc347aa None None] MFA Rules not processed for user `59729b399bed476ea439f1ef28893455`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:24:05.905050 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-ecb2765e-6432-4790-ab72-cde24bc347aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:24:05.981196 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1452/2874] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 16:24:05 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:24:05.994027 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e54c25-77ad-4ec4-ab91-4d59bb7f02d1 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:24:05.995893 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-95e54c25-77ad-4ec4-ab91-4d59bb7f02d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:24:06.015793 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:24:06.061444 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e54c25-77ad-4ec4-ab91-4d59bb7f02d1 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:24:06.065098 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48a986b-3c84-49d4-af3d-1f0b61a8ee2d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3e1a2b83f3d4d88ba5084dfaffd22f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:24:06.065760 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48a986b-3c84-49d4-af3d-1f0b61a8ee2d service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:24:06.066049 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48a986b-3c84-49d4-af3d-1f0b61a8ee2d service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:24:06.066329 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48a986b-3c84-49d4-af3d-1f0b61a8ee2d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:24:06.123179 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48a986b-3c84-49d4-af3d-1f0b61a8ee2d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:24:06.124614 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48a986b-3c84-49d4-af3d-1f0b61a8ee2d service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:24:06.133863 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1423/2875] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:24:05 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:24:07.655910 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb2765e-6432-4790-ab72-cde24bc347aa None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:24:07.657285 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-ecb2765e-6432-4790-ab72-cde24bc347aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:24:07.670080 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb2765e-6432-4790-ab72-cde24bc347aa None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:24:07.673835 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e5dbab-c91b-4cd1-a78a-1a570deeedb2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3e1a2b83f3d4d88ba5084dfaffd22f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:24:07.674494 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e5dbab-c91b-4cd1-a78a-1a570deeedb2 service cinder] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:24:07.674808 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e5dbab-c91b-4cd1-a78a-1a570deeedb2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:24:07.675092 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e5dbab-c91b-4cd1-a78a-1a570deeedb2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:24:07.733173 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e5dbab-c91b-4cd1-a78a-1a570deeedb2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:24:07.734555 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e5dbab-c91b-4cd1-a78a-1a570deeedb2 service cinder] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:24:07.744122 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1453/2876] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:24:07 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:24:08.316823 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48a986b-3c84-49d4-af3d-1f0b61a8ee2d service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:24:08.326426 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48a986b-3c84-49d4-af3d-1f0b61a8ee2d service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:24:08.330867 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee79743-8da6-4ba2-847b-c7a20fcefffb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3e1a2b83f3d4d88ba5084dfaffd22f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:24:08.330867 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee79743-8da6-4ba2-847b-c7a20fcefffb service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:24:08.331219 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee79743-8da6-4ba2-847b-c7a20fcefffb service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:24:08.332647 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee79743-8da6-4ba2-847b-c7a20fcefffb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:24:08.347146 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee79743-8da6-4ba2-847b-c7a20fcefffb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:24:08.348648 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee79743-8da6-4ba2-847b-c7a20fcefffb service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:24:08.363819 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1424/2877] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:24:08 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 16:24:12.308909 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e5dbab-c91b-4cd1-a78a-1a570deeedb2 service cinder] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:24:12.373176 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e5dbab-c91b-4cd1-a78a-1a570deeedb2 service cinder] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:24:12.379649 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e297836-98a6-43c1-9e40-b37878cba66b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:24:12.380554 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e297836-98a6-43c1-9e40-b37878cba66b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:24:12.380838 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e297836-98a6-43c1-9e40-b37878cba66b admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:24:12.381114 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e297836-98a6-43c1-9e40-b37878cba66b admin admin] PATH_INFO: `/v3/users/d3eb51ff6bdf42708dab1cb3e0e861a2` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:24:12.382356 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e297836-98a6-43c1-9e40-b37878cba66b admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3eb51ff6bdf42708dab1cb3e0e861a2)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:24:12.385375 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e297836-98a6-43c1-9e40-b37878cba66b admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:24:12.417254 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:24:12.444191 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-6e297836-98a6-43c1-9e40-b37878cba66b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3eb51ff6bdf42708dab1cb3e0e861a2'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:24:12.452861 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-6e297836-98a6-43c1-9e40-b37878cba66b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3eb51ff6bdf42708dab1cb3e0e861a2'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:24:12.465559 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-6e297836-98a6-43c1-9e40-b37878cba66b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3eb51ff6bdf42708dab1cb3e0e861a2'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:24:12.466239 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-6e297836-98a6-43c1-9e40-b37878cba66b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3eb51ff6bdf42708dab1cb3e0e861a2'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:24:12.466838 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-6e297836-98a6-43c1-9e40-b37878cba66b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3eb51ff6bdf42708dab1cb3e0e861a2'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:24:12.479848 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1454/2878] 10.222.0.99 () {58 vars in 1223 bytes} [Thu Jul 14 16:24:12 2022] DELETE /identity/v3/users/d3eb51ff6bdf42708dab1cb3e0e861a2 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:24:12.484536 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c51fdce-7b3d-49a3-8625-6542dffa7135 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:24:12.485001 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c51fdce-7b3d-49a3-8625-6542dffa7135 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:24:12.485088 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c51fdce-7b3d-49a3-8625-6542dffa7135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:24:12.504099 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-8c51fdce-7b3d-49a3-8625-6542dffa7135 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:24:12.505147 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-8c51fdce-7b3d-49a3-8625-6542dffa7135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:24:12.568511 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1425/2879] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 16:24:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:24:12.581164 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e297836-98a6-43c1-9e40-b37878cba66b admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:24:12.582664 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-6e297836-98a6-43c1-9e40-b37878cba66b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:24:12.643678 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e297836-98a6-43c1-9e40-b37878cba66b admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:24:12.647555 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a2103f-648a-4de2-b23f-61d6a312f90c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:24:12.649474 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a2103f-648a-4de2-b23f-61d6a312f90c service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:24:12.649781 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a2103f-648a-4de2-b23f-61d6a312f90c service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:24:12.650057 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a2103f-648a-4de2-b23f-61d6a312f90c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:24:12.660542 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a2103f-648a-4de2-b23f-61d6a312f90c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:24:12.662148 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a2103f-648a-4de2-b23f-61d6a312f90c service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:24:12.673581 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1455/2880] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:24:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:24:12.995318 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c51fdce-7b3d-49a3-8625-6542dffa7135 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:24:13.002170 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-8c51fdce-7b3d-49a3-8625-6542dffa7135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:24:13.017654 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c51fdce-7b3d-49a3-8625-6542dffa7135 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:24:13.026034 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82e7455-31b0-4d59-b30c-f9a588dade20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:24:13.027611 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82e7455-31b0-4d59-b30c-f9a588dade20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:24:13.028147 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82e7455-31b0-4d59-b30c-f9a588dade20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:24:13.028744 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82e7455-31b0-4d59-b30c-f9a588dade20 admin admin] PATH_INFO: `/v3/projects/588fcdcf9d34452889ebd1184ea6c62a` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:24:13.030970 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82e7455-31b0-4d59-b30c-f9a588dade20 admin admin] RBAC: Authorizing `identity:delete_project(project_id=588fcdcf9d34452889ebd1184ea6c62a)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:24:13.033533 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82e7455-31b0-4d59-b30c-f9a588dade20 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:24:13.088191 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-b82e7455-31b0-4d59-b30c-f9a588dade20 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '588fcdcf9d34452889ebd1184ea6c62a', 'request_id': 'req-b82e7455-31b0-4d59-b30c-f9a588dade20'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:24:13.090816 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:24:13.096057 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-b82e7455-31b0-4d59-b30c-f9a588dade20 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '588fcdcf9d34452889ebd1184ea6c62a', 'request_id': 'req-b82e7455-31b0-4d59-b30c-f9a588dade20'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:24:13.103833 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-b82e7455-31b0-4d59-b30c-f9a588dade20 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '588fcdcf9d34452889ebd1184ea6c62a', 'request_id': 'req-b82e7455-31b0-4d59-b30c-f9a588dade20'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:24:13.118987 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-b82e7455-31b0-4d59-b30c-f9a588dade20 admin admin] The token cache is being invalidate because project 588fcdcf9d34452889ebd1184ea6c62a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 16:24:13.119490 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-b82e7455-31b0-4d59-b30c-f9a588dade20 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:24:13.120203 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-b82e7455-31b0-4d59-b30c-f9a588dade20 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:24:13.125688 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1426/2881] 10.222.0.99 () {58 vars in 1232 bytes} [Thu Jul 14 16:24:12 2022] DELETE /identity/v3/projects/588fcdcf9d34452889ebd1184ea6c62a => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:24:13.127330 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a2103f-648a-4de2-b23f-61d6a312f90c service neutron] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:24:13.138844 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a2103f-648a-4de2-b23f-61d6a312f90c service neutron] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:24:13.222503 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9954cf2-8667-46d4-b084-688e9c4f8942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:24:13.223391 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9954cf2-8667-46d4-b084-688e9c4f8942 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:24:13.223708 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9954cf2-8667-46d4-b084-688e9c4f8942 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:24:13.223993 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9954cf2-8667-46d4-b084-688e9c4f8942 admin admin] PATH_INFO: `/v3/users/55545650385f4b98a8f25c0e0949062c` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:24:13.225255 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9954cf2-8667-46d4-b084-688e9c4f8942 admin admin] RBAC: Authorizing `identity:delete_user(user_id=55545650385f4b98a8f25c0e0949062c)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:24:13.228276 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9954cf2-8667-46d4-b084-688e9c4f8942 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:24:13.291958 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-f9954cf2-8667-46d4-b084-688e9c4f8942 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55545650385f4b98a8f25c0e0949062c'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:24:13.294217 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:24:13.303856 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-f9954cf2-8667-46d4-b084-688e9c4f8942 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55545650385f4b98a8f25c0e0949062c'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:24:13.320412 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-f9954cf2-8667-46d4-b084-688e9c4f8942 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55545650385f4b98a8f25c0e0949062c'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:24:13.321145 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-f9954cf2-8667-46d4-b084-688e9c4f8942 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55545650385f4b98a8f25c0e0949062c'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:24:13.321751 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-f9954cf2-8667-46d4-b084-688e9c4f8942 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55545650385f4b98a8f25c0e0949062c'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:24:13.340261 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1456/2882] 10.222.0.99 () {58 vars in 1223 bytes} [Thu Jul 14 16:24:13 2022] DELETE /identity/v3/users/55545650385f4b98a8f25c0e0949062c => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:24:13.666555 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82e7455-31b0-4d59-b30c-f9a588dade20 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:24:13.668069 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b82e7455-31b0-4d59-b30c-f9a588dade20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:24:13.685528 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82e7455-31b0-4d59-b30c-f9a588dade20 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:24:13.761641 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5a3888-28b0-4c57-ae25-3e93dc1946df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:24:13.762512 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5a3888-28b0-4c57-ae25-3e93dc1946df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:24:13.762797 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5a3888-28b0-4c57-ae25-3e93dc1946df admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:24:13.763078 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5a3888-28b0-4c57-ae25-3e93dc1946df admin admin] PATH_INFO: `/v3/projects/b8b0fb6310ef4f4092511cceb2b7b2ec` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:24:13.764280 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5a3888-28b0-4c57-ae25-3e93dc1946df admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8b0fb6310ef4f4092511cceb2b7b2ec)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:24:13.767318 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5a3888-28b0-4c57-ae25-3e93dc1946df admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:24:13.822217 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4c5a3888-28b0-4c57-ae25-3e93dc1946df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8b0fb6310ef4f4092511cceb2b7b2ec', 'request_id': 'req-4c5a3888-28b0-4c57-ae25-3e93dc1946df'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:24:13.828904 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4c5a3888-28b0-4c57-ae25-3e93dc1946df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8b0fb6310ef4f4092511cceb2b7b2ec', 'request_id': 'req-4c5a3888-28b0-4c57-ae25-3e93dc1946df'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:24:13.834764 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4c5a3888-28b0-4c57-ae25-3e93dc1946df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8b0fb6310ef4f4092511cceb2b7b2ec', 'request_id': 'req-4c5a3888-28b0-4c57-ae25-3e93dc1946df'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:24:13.854563 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4c5a3888-28b0-4c57-ae25-3e93dc1946df admin admin] The token cache is being invalidate because project b8b0fb6310ef4f4092511cceb2b7b2ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 16:24:13.854855 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4c5a3888-28b0-4c57-ae25-3e93dc1946df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:24:13.855510 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4c5a3888-28b0-4c57-ae25-3e93dc1946df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:24:13.857273 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1427/2883] 10.222.0.99 () {58 vars in 1232 bytes} [Thu Jul 14 16:24:13 2022] DELETE /identity/v3/projects/b8b0fb6310ef4f4092511cceb2b7b2ec => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:24:13.861554 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9954cf2-8667-46d4-b084-688e9c4f8942 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:24:13.863134 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-f9954cf2-8667-46d4-b084-688e9c4f8942 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:24:13.876897 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9954cf2-8667-46d4-b084-688e9c4f8942 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:24:13.938665 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f8c596-33c4-41c6-b52d-2a238a5d5d56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:24:13.939550 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f8c596-33c4-41c6-b52d-2a238a5d5d56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:24:13.939914 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f8c596-33c4-41c6-b52d-2a238a5d5d56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:24:13.940199 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f8c596-33c4-41c6-b52d-2a238a5d5d56 admin admin] PATH_INFO: `/v3/users/59729b399bed476ea439f1ef28893455` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:24:13.941404 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f8c596-33c4-41c6-b52d-2a238a5d5d56 admin admin] RBAC: Authorizing `identity:delete_user(user_id=59729b399bed476ea439f1ef28893455)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:24:13.943322 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f8c596-33c4-41c6-b52d-2a238a5d5d56 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:24:14.002991 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-63f8c596-33c4-41c6-b52d-2a238a5d5d56 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59729b399bed476ea439f1ef28893455'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:24:14.012664 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-63f8c596-33c4-41c6-b52d-2a238a5d5d56 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59729b399bed476ea439f1ef28893455'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:24:14.029423 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-63f8c596-33c4-41c6-b52d-2a238a5d5d56 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59729b399bed476ea439f1ef28893455'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:24:14.030161 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-63f8c596-33c4-41c6-b52d-2a238a5d5d56 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59729b399bed476ea439f1ef28893455'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:24:14.030837 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-63f8c596-33c4-41c6-b52d-2a238a5d5d56 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59729b399bed476ea439f1ef28893455'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:24:14.045913 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1457/2884] 10.222.0.99 () {58 vars in 1223 bytes} [Thu Jul 14 16:24:13 2022] DELETE /identity/v3/users/59729b399bed476ea439f1ef28893455 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:24:14.365857 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5a3888-28b0-4c57-ae25-3e93dc1946df admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:24:14.368627 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-4c5a3888-28b0-4c57-ae25-3e93dc1946df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:24:14.384281 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5a3888-28b0-4c57-ae25-3e93dc1946df admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:24:14.448899 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7087cd-3ed6-45f4-b9fc-101033186dee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:24:14.449723 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7087cd-3ed6-45f4-b9fc-101033186dee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:24:14.450006 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7087cd-3ed6-45f4-b9fc-101033186dee admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:24:14.450289 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7087cd-3ed6-45f4-b9fc-101033186dee admin admin] PATH_INFO: `/v3/projects/17a1648f4a8d4babaef43532d9f67c49` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:24:14.452741 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7087cd-3ed6-45f4-b9fc-101033186dee admin admin] RBAC: Authorizing `identity:delete_project(project_id=17a1648f4a8d4babaef43532d9f67c49)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:24:14.454644 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7087cd-3ed6-45f4-b9fc-101033186dee admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:24:14.468186 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:24:14.509646 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-fc7087cd-3ed6-45f4-b9fc-101033186dee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17a1648f4a8d4babaef43532d9f67c49', 'request_id': 'req-fc7087cd-3ed6-45f4-b9fc-101033186dee'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:24:14.516888 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-fc7087cd-3ed6-45f4-b9fc-101033186dee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17a1648f4a8d4babaef43532d9f67c49', 'request_id': 'req-fc7087cd-3ed6-45f4-b9fc-101033186dee'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:24:14.524066 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-fc7087cd-3ed6-45f4-b9fc-101033186dee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17a1648f4a8d4babaef43532d9f67c49', 'request_id': 'req-fc7087cd-3ed6-45f4-b9fc-101033186dee'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:24:14.538120 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-fc7087cd-3ed6-45f4-b9fc-101033186dee admin admin] The token cache is being invalidate because project 17a1648f4a8d4babaef43532d9f67c49 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 16:24:14.539119 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-fc7087cd-3ed6-45f4-b9fc-101033186dee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:24:14.539797 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-fc7087cd-3ed6-45f4-b9fc-101033186dee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:24:14.541411 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1428/2885] 10.222.0.99 () {58 vars in 1232 bytes} [Thu Jul 14 16:24:14 2022] DELETE /identity/v3/projects/17a1648f4a8d4babaef43532d9f67c49 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:24:14.546538 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18ededc-b44b-4348-b989-1d9083c157c5 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:24:14.547280 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18ededc-b44b-4348-b989-1d9083c157c5 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:24:14.547280 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18ededc-b44b-4348-b989-1d9083c157c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:24:14.567569 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-d18ededc-b44b-4348-b989-1d9083c157c5 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:24:14.568812 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-d18ededc-b44b-4348-b989-1d9083c157c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:24:14.613958 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:24:14.638245 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1458/2886] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 16:24:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:24:14.646538 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ead28ca-5148-4c37-ac28-95817a1cbc35 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:24:14.646828 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ead28ca-5148-4c37-ac28-95817a1cbc35 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:24:14.647111 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ead28ca-5148-4c37-ac28-95817a1cbc35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:24:14.667923 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-4ead28ca-5148-4c37-ac28-95817a1cbc35 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:24:14.669293 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-4ead28ca-5148-4c37-ac28-95817a1cbc35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:24:14.677948 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1429/2887] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 16:24:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:24:14.683464 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18ededc-b44b-4348-b989-1d9083c157c5 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:24:14.693994 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18ededc-b44b-4348-b989-1d9083c157c5 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:24:14.699401 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6f8ada-72d2-4710-a368-e79b3554fa38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:24:14.700052 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6f8ada-72d2-4710-a368-e79b3554fa38 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:24:14.700342 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6f8ada-72d2-4710-a368-e79b3554fa38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:24:14.700626 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6f8ada-72d2-4710-a368-e79b3554fa38 admin admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:24:14.701394 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6f8ada-72d2-4710-a368-e79b3554fa38 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 16:24:14.701796 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6f8ada-72d2-4710-a368-e79b3554fa38 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:24:14.703432 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6f8ada-72d2-4710-a368-e79b3554fa38 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:24:14.718197 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1459/2888] 10.222.0.99 () {58 vars in 1152 bytes} [Thu Jul 14 16:24:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 16:24:14.722790 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ead28ca-5148-4c37-ac28-95817a1cbc35 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:24:14.734988 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ead28ca-5148-4c37-ac28-95817a1cbc35 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:24:14.739862 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76d72f4-670f-4e95-875b-13440d563f49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:24:14.740527 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76d72f4-670f-4e95-875b-13440d563f49 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:24:14.740817 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76d72f4-670f-4e95-875b-13440d563f49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:24:14.741128 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76d72f4-670f-4e95-875b-13440d563f49 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:24:14.742050 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76d72f4-670f-4e95-875b-13440d563f49 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:24:14.744068 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76d72f4-670f-4e95-875b-13440d563f49 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:24:14.776842 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1430/2889] 10.222.0.99 () {60 vars in 1152 bytes} [Thu Jul 14 16:24:14 2022] POST /identity/v3/projects => generated 369 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 16:24:14.781539 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6f8ada-72d2-4710-a368-e79b3554fa38 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:24:14.792842 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6f8ada-72d2-4710-a368-e79b3554fa38 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:24:14.860139 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266b28f1-903a-48c7-bcc2-98a18bd98314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:24:14.861923 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266b28f1-903a-48c7-bcc2-98a18bd98314 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:24:14.863052 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266b28f1-903a-48c7-bcc2-98a18bd98314 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:24:14.863358 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266b28f1-903a-48c7-bcc2-98a18bd98314 admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:24:14.867643 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266b28f1-903a-48c7-bcc2-98a18bd98314 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:24:14.869571 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266b28f1-903a-48c7-bcc2-98a18bd98314 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:24:14.898909 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1460/2890] 10.222.0.99 () {60 vars in 1143 bytes} [Thu Jul 14 16:24:14 2022] POST /identity/v3/users => generated 285 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 16:24:14.903516 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76d72f4-670f-4e95-875b-13440d563f49 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:24:14.917988 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76d72f4-670f-4e95-875b-13440d563f49 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:24:14.923546 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aae6feb-1d70-49a9-9330-964b55796868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:24:14.924274 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aae6feb-1d70-49a9-9330-964b55796868 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:24:14.924570 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aae6feb-1d70-49a9-9330-964b55796868 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:24:14.924857 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aae6feb-1d70-49a9-9330-964b55796868 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:24:14.926518 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aae6feb-1d70-49a9-9330-964b55796868 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:24:14.928524 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aae6feb-1d70-49a9-9330-964b55796868 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:24:14.941424 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1431/2891] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 16:24:14 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:24:14.946811 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266b28f1-903a-48c7-bcc2-98a18bd98314 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:24:14.963633 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266b28f1-903a-48c7-bcc2-98a18bd98314 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:24:14.969175 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f6e117-7c2d-4186-b675-fb9401bb9a55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:24:14.969962 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f6e117-7c2d-4186-b675-fb9401bb9a55 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:24:14.970252 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f6e117-7c2d-4186-b675-fb9401bb9a55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:24:14.970536 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f6e117-7c2d-4186-b675-fb9401bb9a55 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:24:14.971278 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f6e117-7c2d-4186-b675-fb9401bb9a55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:24:14.973015 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f6e117-7c2d-4186-b675-fb9401bb9a55 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:24:14.987522 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1461/2892] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 16:24:14 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:24:14.992599 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aae6feb-1d70-49a9-9330-964b55796868 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:24:15.012831 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aae6feb-1d70-49a9-9330-964b55796868 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:24:15.019790 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39af545-c360-40ff-b006-85b05c4a275a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:24:15.021581 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39af545-c360-40ff-b006-85b05c4a275a admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:24:15.021979 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39af545-c360-40ff-b006-85b05c4a275a admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:24:15.023354 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39af545-c360-40ff-b006-85b05c4a275a admin admin] PATH_INFO: `/v3/projects/98635c67a729432590aa32c9091a2f97/users/e3769d5c24b8483389a5fcdc1e3595e7/roles/b8c98fddc0fd4436b577fd1d3361bc38` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:24:15.044851 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39af545-c360-40ff-b006-85b05c4a275a admin admin] RBAC: Authorizing `identity:create_grant(project_id=98635c67a729432590aa32c9091a2f97, user_id=e3769d5c24b8483389a5fcdc1e3595e7, role_id=b8c98fddc0fd4436b577fd1d3361bc38)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:24:15.048480 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39af545-c360-40ff-b006-85b05c4a275a admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:24:15.066052 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1432/2893] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 16:24:14 2022] PUT /identity/v3/projects/98635c67a729432590aa32c9091a2f97/users/e3769d5c24b8483389a5fcdc1e3595e7/roles/b8c98fddc0fd4436b577fd1d3361bc38 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:24:15.078234 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae865c5e-9d30-4319-a170-2b1449f912a8 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:24:15.078684 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae865c5e-9d30-4319-a170-2b1449f912a8 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:24:15.079167 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae865c5e-9d30-4319-a170-2b1449f912a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:24:15.113223 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-ae865c5e-9d30-4319-a170-2b1449f912a8 None None] MFA Rules not processed for user `e3769d5c24b8483389a5fcdc1e3595e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:24:15.114442 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-ae865c5e-9d30-4319-a170-2b1449f912a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:24:15.184467 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1462/2894] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 16:24:15 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:24:15.196272 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39af545-c360-40ff-b006-85b05c4a275a admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:24:15.198569 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a39af545-c360-40ff-b006-85b05c4a275a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:24:15.266303 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39af545-c360-40ff-b006-85b05c4a275a admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:24:15.270001 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96232789-f67b-4e42-9e1e-7601441600a4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605e230d50ce44629f554c95b2f79ead', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:24:15.270654 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96232789-f67b-4e42-9e1e-7601441600a4 service glance] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:24:15.270961 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96232789-f67b-4e42-9e1e-7601441600a4 service glance] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:24:15.271241 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96232789-f67b-4e42-9e1e-7601441600a4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:24:15.281358 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96232789-f67b-4e42-9e1e-7601441600a4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:24:15.282799 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96232789-f67b-4e42-9e1e-7601441600a4 service glance] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:24:15.293463 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1433/2895] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:24:15 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:24:25.180474 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=79,keystone:INSERT=3 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:24:25.287963 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=72,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:24:49.892529 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae865c5e-9d30-4319-a170-2b1449f912a8 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:24:49.955445 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae865c5e-9d30-4319-a170-2b1449f912a8 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:24:49.960195 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a871fea-664f-4963-bcc0-0a72f7b4826a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:24:49.961087 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a871fea-664f-4963-bcc0-0a72f7b4826a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:24:49.961353 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a871fea-664f-4963-bcc0-0a72f7b4826a admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:24:49.961353 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a871fea-664f-4963-bcc0-0a72f7b4826a admin admin] PATH_INFO: `/v3/users/e3769d5c24b8483389a5fcdc1e3595e7` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:24:49.962821 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a871fea-664f-4963-bcc0-0a72f7b4826a admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3769d5c24b8483389a5fcdc1e3595e7)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:24:49.964858 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a871fea-664f-4963-bcc0-0a72f7b4826a admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:24:50.019884 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-6a871fea-664f-4963-bcc0-0a72f7b4826a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3769d5c24b8483389a5fcdc1e3595e7'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:24:50.028498 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-6a871fea-664f-4963-bcc0-0a72f7b4826a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3769d5c24b8483389a5fcdc1e3595e7'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:24:50.041127 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-6a871fea-664f-4963-bcc0-0a72f7b4826a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3769d5c24b8483389a5fcdc1e3595e7'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:24:50.042037 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-6a871fea-664f-4963-bcc0-0a72f7b4826a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3769d5c24b8483389a5fcdc1e3595e7'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:24:50.042714 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-6a871fea-664f-4963-bcc0-0a72f7b4826a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3769d5c24b8483389a5fcdc1e3595e7'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:24:50.056666 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1463/2896] 10.222.0.99 () {58 vars in 1223 bytes} [Thu Jul 14 16:24:49 2022] DELETE /identity/v3/users/e3769d5c24b8483389a5fcdc1e3595e7 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:24:50.061153 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5be6aa-49f0-4752-946d-1dc84f72b7c6 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:24:50.061640 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5be6aa-49f0-4752-946d-1dc84f72b7c6 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:24:50.061742 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5be6aa-49f0-4752-946d-1dc84f72b7c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:24:50.089086 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-5a5be6aa-49f0-4752-946d-1dc84f72b7c6 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:24:50.090304 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-5a5be6aa-49f0-4752-946d-1dc84f72b7c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:24:50.151378 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1434/2897] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 16:24:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:24:50.161055 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a871fea-664f-4963-bcc0-0a72f7b4826a admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:24:50.162494 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-6a871fea-664f-4963-bcc0-0a72f7b4826a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:24:50.221832 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a871fea-664f-4963-bcc0-0a72f7b4826a admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:24:50.225575 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ca2ad9-a2d4-4a53-b96b-98d89dedd7dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:24:50.226253 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ca2ad9-a2d4-4a53-b96b-98d89dedd7dd service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:24:50.226546 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ca2ad9-a2d4-4a53-b96b-98d89dedd7dd service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:24:50.227490 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ca2ad9-a2d4-4a53-b96b-98d89dedd7dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:24:50.236820 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ca2ad9-a2d4-4a53-b96b-98d89dedd7dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:24:50.238264 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ca2ad9-a2d4-4a53-b96b-98d89dedd7dd service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:24:50.248810 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1464/2898] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:24:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:24:50.519779 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5be6aa-49f0-4752-946d-1dc84f72b7c6 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:24:50.521290 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-5a5be6aa-49f0-4752-946d-1dc84f72b7c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:24:50.533662 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5be6aa-49f0-4752-946d-1dc84f72b7c6 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:24:50.538366 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9741da37-9633-4ab2-9ae3-1a4689c2c4a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:24:50.540031 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9741da37-9633-4ab2-9ae3-1a4689c2c4a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:24:50.540400 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9741da37-9633-4ab2-9ae3-1a4689c2c4a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:24:50.540737 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9741da37-9633-4ab2-9ae3-1a4689c2c4a4 admin admin] PATH_INFO: `/v3/projects/98635c67a729432590aa32c9091a2f97` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:24:50.542895 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9741da37-9633-4ab2-9ae3-1a4689c2c4a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=98635c67a729432590aa32c9091a2f97)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:24:50.544893 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9741da37-9633-4ab2-9ae3-1a4689c2c4a4 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:24:50.594799 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-9741da37-9633-4ab2-9ae3-1a4689c2c4a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98635c67a729432590aa32c9091a2f97', 'request_id': 'req-9741da37-9633-4ab2-9ae3-1a4689c2c4a4'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:24:50.601046 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-9741da37-9633-4ab2-9ae3-1a4689c2c4a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98635c67a729432590aa32c9091a2f97', 'request_id': 'req-9741da37-9633-4ab2-9ae3-1a4689c2c4a4'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:24:50.607408 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-9741da37-9633-4ab2-9ae3-1a4689c2c4a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98635c67a729432590aa32c9091a2f97', 'request_id': 'req-9741da37-9633-4ab2-9ae3-1a4689c2c4a4'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:24:50.620231 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-9741da37-9633-4ab2-9ae3-1a4689c2c4a4 admin admin] The token cache is being invalidate because project 98635c67a729432590aa32c9091a2f97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 16:24:50.620525 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-9741da37-9633-4ab2-9ae3-1a4689c2c4a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:24:50.621162 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-9741da37-9633-4ab2-9ae3-1a4689c2c4a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:24:50.622696 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1435/2899] 10.222.0.99 () {58 vars in 1232 bytes} [Thu Jul 14 16:24:50 2022] DELETE /identity/v3/projects/98635c67a729432590aa32c9091a2f97 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:24:50.627580 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747ac443-64c7-468e-b6dc-a0d2fdf07e88 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:24:50.627905 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747ac443-64c7-468e-b6dc-a0d2fdf07e88 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:24:50.628189 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747ac443-64c7-468e-b6dc-a0d2fdf07e88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:24:50.646727 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-747ac443-64c7-468e-b6dc-a0d2fdf07e88 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:24:50.647998 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-747ac443-64c7-468e-b6dc-a0d2fdf07e88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:24:50.666921 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:24:50.712628 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1465/2900] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 16:24:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 16:24:50.720232 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3a3ee6-ef63-420d-810b-7c0c6be9bf61 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:24:50.720528 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3a3ee6-ef63-420d-810b-7c0c6be9bf61 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:24:50.720807 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3a3ee6-ef63-420d-810b-7c0c6be9bf61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:24:50.740286 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-cd3a3ee6-ef63-420d-810b-7c0c6be9bf61 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:24:50.741497 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-cd3a3ee6-ef63-420d-810b-7c0c6be9bf61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:24:50.749487 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1436/2901] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 16:24:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:24:50.753823 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747ac443-64c7-468e-b6dc-a0d2fdf07e88 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:24:50.764207 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747ac443-64c7-468e-b6dc-a0d2fdf07e88 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:24:50.768980 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e4c4e4-2034-4b53-8337-4afe3a1bd7b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:24:50.769650 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e4c4e4-2034-4b53-8337-4afe3a1bd7b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:24:50.769939 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e4c4e4-2034-4b53-8337-4afe3a1bd7b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:24:50.770224 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e4c4e4-2034-4b53-8337-4afe3a1bd7b1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:24:50.770961 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e4c4e4-2034-4b53-8337-4afe3a1bd7b1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80236) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 16:24:50.771283 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e4c4e4-2034-4b53-8337-4afe3a1bd7b1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:24:50.772998 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e4c4e4-2034-4b53-8337-4afe3a1bd7b1 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:24:50.785661 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1466/2902] 10.222.0.99 () {58 vars in 1152 bytes} [Thu Jul 14 16:24:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 16:24:50.790036 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3a3ee6-ef63-420d-810b-7c0c6be9bf61 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:24:50.799093 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3a3ee6-ef63-420d-810b-7c0c6be9bf61 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:24:50.803737 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c85941-9e82-4127-a104-98b67f0c7553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:24:50.804367 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c85941-9e82-4127-a104-98b67f0c7553 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:24:50.804945 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c85941-9e82-4127-a104-98b67f0c7553 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:24:50.805220 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c85941-9e82-4127-a104-98b67f0c7553 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:24:50.806018 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c85941-9e82-4127-a104-98b67f0c7553 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:24:50.807921 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c85941-9e82-4127-a104-98b67f0c7553 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:24:50.837807 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1437/2903] 10.222.0.99 () {60 vars in 1152 bytes} [Thu Jul 14 16:24:50 2022] POST /identity/v3/projects => generated 369 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 16:24:50.842074 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e4c4e4-2034-4b53-8337-4afe3a1bd7b1 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:24:50.852872 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e4c4e4-2034-4b53-8337-4afe3a1bd7b1 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:24:50.909865 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd1cf37-102e-45cc-9ac6-2c76c15c0013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:24:50.910545 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd1cf37-102e-45cc-9ac6-2c76c15c0013 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:24:50.910828 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd1cf37-102e-45cc-9ac6-2c76c15c0013 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:24:50.911110 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd1cf37-102e-45cc-9ac6-2c76c15c0013 admin admin] PATH_INFO: `/v3/users` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:24:50.911848 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd1cf37-102e-45cc-9ac6-2c76c15c0013 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:24:50.913768 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd1cf37-102e-45cc-9ac6-2c76c15c0013 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:24:50.936620 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1467/2904] 10.222.0.99 () {60 vars in 1143 bytes} [Thu Jul 14 16:24:50 2022] POST /identity/v3/users => generated 285 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 16:24:50.940818 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c85941-9e82-4127-a104-98b67f0c7553 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:24:50.960323 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c85941-9e82-4127-a104-98b67f0c7553 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:24:50.960323 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e340b23f-ad1a-40bf-bacf-98be4528b871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:24:50.960323 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e340b23f-ad1a-40bf-bacf-98be4528b871 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:24:50.960323 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e340b23f-ad1a-40bf-bacf-98be4528b871 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:24:50.960751 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e340b23f-ad1a-40bf-bacf-98be4528b871 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:24:50.960751 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e340b23f-ad1a-40bf-bacf-98be4528b871 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:24:50.960751 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e340b23f-ad1a-40bf-bacf-98be4528b871 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:24:50.971091 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1438/2905] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 16:24:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 16:24:50.975149 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd1cf37-102e-45cc-9ac6-2c76c15c0013 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:24:50.985203 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd1cf37-102e-45cc-9ac6-2c76c15c0013 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:24:50.989926 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05478c29-2dd2-412b-9ee3-c3b22049c037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:24:50.990598 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05478c29-2dd2-412b-9ee3-c3b22049c037 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:24:50.991067 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05478c29-2dd2-412b-9ee3-c3b22049c037 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:24:50.991343 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05478c29-2dd2-412b-9ee3-c3b22049c037 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:24:50.992172 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05478c29-2dd2-412b-9ee3-c3b22049c037 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:24:50.993882 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05478c29-2dd2-412b-9ee3-c3b22049c037 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:24:51.004587 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1468/2906] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 16:24:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:24:51.009604 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e340b23f-ad1a-40bf-bacf-98be4528b871 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:24:51.019565 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e340b23f-ad1a-40bf-bacf-98be4528b871 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:24:51.025836 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207acb96-b8cf-4d68-8239-b44598839c5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:24:51.026880 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207acb96-b8cf-4d68-8239-b44598839c5a admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:24:51.027113 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207acb96-b8cf-4d68-8239-b44598839c5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:24:51.027415 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207acb96-b8cf-4d68-8239-b44598839c5a admin admin] PATH_INFO: `/v3/projects/3dbb1f888fb3409e93187a867375fb82/users/fadbd0a92df24d3c880fcee4093aacdc/roles/b8c98fddc0fd4436b577fd1d3361bc38` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:24:51.042160 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207acb96-b8cf-4d68-8239-b44598839c5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3dbb1f888fb3409e93187a867375fb82, user_id=fadbd0a92df24d3c880fcee4093aacdc, role_id=b8c98fddc0fd4436b577fd1d3361bc38)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:24:51.044841 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207acb96-b8cf-4d68-8239-b44598839c5a admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:24:51.059559 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1439/2907] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 16:24:51 2022] PUT /identity/v3/projects/3dbb1f888fb3409e93187a867375fb82/users/fadbd0a92df24d3c880fcee4093aacdc/roles/b8c98fddc0fd4436b577fd1d3361bc38 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:24:51.070032 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4266dbc0-dcdb-497a-97d5-1ef97820aa95 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:24:51.070418 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4266dbc0-dcdb-497a-97d5-1ef97820aa95 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:24:51.071196 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4266dbc0-dcdb-497a-97d5-1ef97820aa95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:24:51.097570 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-4266dbc0-dcdb-497a-97d5-1ef97820aa95 None None] MFA Rules not processed for user `fadbd0a92df24d3c880fcee4093aacdc`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:24:51.098793 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-4266dbc0-dcdb-497a-97d5-1ef97820aa95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:24:51.171990 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1469/2908] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 16:24:51 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:24:51.183967 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207acb96-b8cf-4d68-8239-b44598839c5a admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:24:51.185856 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-207acb96-b8cf-4d68-8239-b44598839c5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:24:51.200386 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:24:51.247749 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207acb96-b8cf-4d68-8239-b44598839c5a admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:24:51.252359 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c364d902-6ea4-4d4b-9f31-e87df76d036e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605e230d50ce44629f554c95b2f79ead', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:24:51.253135 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c364d902-6ea4-4d4b-9f31-e87df76d036e service glance] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:24:51.253425 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c364d902-6ea4-4d4b-9f31-e87df76d036e service glance] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:24:51.253711 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c364d902-6ea4-4d4b-9f31-e87df76d036e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:24:51.265715 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c364d902-6ea4-4d4b-9f31-e87df76d036e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:24:51.267176 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c364d902-6ea4-4d4b-9f31-e87df76d036e service glance] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:24:51.277323 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1440/2909] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:24:51 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 16:25:01.166728 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:25:01.272252 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=21 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:25:10.634317 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4266dbc0-dcdb-497a-97d5-1ef97820aa95 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:25:10.635963 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-4266dbc0-dcdb-497a-97d5-1ef97820aa95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:25:10.740850 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4266dbc0-dcdb-497a-97d5-1ef97820aa95 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:25:10.746607 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8421d651-84eb-42b6-b28c-d95894dd34c1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc157c485f794e968a60d8405c9310ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:25:10.747259 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8421d651-84eb-42b6-b28c-d95894dd34c1 service placement] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:25:10.747544 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8421d651-84eb-42b6-b28c-d95894dd34c1 service placement] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:25:10.747866 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8421d651-84eb-42b6-b28c-d95894dd34c1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:25:10.749645 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-8421d651-84eb-42b6-b28c-d95894dd34c1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:25:10.764135 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8421d651-84eb-42b6-b28c-d95894dd34c1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:25:10.765878 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8421d651-84eb-42b6-b28c-d95894dd34c1 service placement] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:25:10.783330 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1470/2910] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:25:10 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:25:20.778493 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=41 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:25:22.150324 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c364d902-6ea4-4d4b-9f31-e87df76d036e service glance] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:25:22.215421 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c364d902-6ea4-4d4b-9f31-e87df76d036e service glance] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:25:22.220023 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c80abe5-d00f-480c-8c9c-1344204fb1f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:25:22.220895 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c80abe5-d00f-480c-8c9c-1344204fb1f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:25:22.221184 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c80abe5-d00f-480c-8c9c-1344204fb1f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:25:22.221466 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c80abe5-d00f-480c-8c9c-1344204fb1f2 admin admin] PATH_INFO: `/v3/users/fadbd0a92df24d3c880fcee4093aacdc` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:25:22.222652 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c80abe5-d00f-480c-8c9c-1344204fb1f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fadbd0a92df24d3c880fcee4093aacdc)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:25:22.224685 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c80abe5-d00f-480c-8c9c-1344204fb1f2 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:25:22.275536 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-0c80abe5-d00f-480c-8c9c-1344204fb1f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fadbd0a92df24d3c880fcee4093aacdc'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:25:22.282908 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-0c80abe5-d00f-480c-8c9c-1344204fb1f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fadbd0a92df24d3c880fcee4093aacdc'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:25:22.297470 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-0c80abe5-d00f-480c-8c9c-1344204fb1f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fadbd0a92df24d3c880fcee4093aacdc'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:25:22.298214 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-0c80abe5-d00f-480c-8c9c-1344204fb1f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fadbd0a92df24d3c880fcee4093aacdc'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:25:22.298859 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-0c80abe5-d00f-480c-8c9c-1344204fb1f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fadbd0a92df24d3c880fcee4093aacdc'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:25:22.313112 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1441/2911] 10.222.0.99 () {58 vars in 1223 bytes} [Thu Jul 14 16:25:22 2022] DELETE /identity/v3/users/fadbd0a92df24d3c880fcee4093aacdc => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:25:22.317577 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65144877-8aee-4746-bf85-e9a8ea1737e4 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:25:22.318105 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65144877-8aee-4746-bf85-e9a8ea1737e4 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:25:22.318263 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65144877-8aee-4746-bf85-e9a8ea1737e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:25:22.336139 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-65144877-8aee-4746-bf85-e9a8ea1737e4 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:25:22.337359 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-65144877-8aee-4746-bf85-e9a8ea1737e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:25:22.401993 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1471/2912] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 16:25:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:25:22.411724 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c80abe5-d00f-480c-8c9c-1344204fb1f2 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:25:22.413863 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0c80abe5-d00f-480c-8c9c-1344204fb1f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:25:22.471831 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c80abe5-d00f-480c-8c9c-1344204fb1f2 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:25:22.476256 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b82fcb6-2905-4a17-ba09-e93718c64ac2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:25:22.476947 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b82fcb6-2905-4a17-ba09-e93718c64ac2 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:25:22.477238 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b82fcb6-2905-4a17-ba09-e93718c64ac2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:25:22.477549 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b82fcb6-2905-4a17-ba09-e93718c64ac2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:25:22.486907 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b82fcb6-2905-4a17-ba09-e93718c64ac2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:25:22.488384 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b82fcb6-2905-4a17-ba09-e93718c64ac2 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:25:22.498550 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1442/2913] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:25:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:25:22.773575 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65144877-8aee-4746-bf85-e9a8ea1737e4 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:25:22.775265 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-65144877-8aee-4746-bf85-e9a8ea1737e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:25:22.787936 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65144877-8aee-4746-bf85-e9a8ea1737e4 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:25:22.793530 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6b77ce-4d59-4731-8351-9865e121ffbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:25:22.799756 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6b77ce-4d59-4731-8351-9865e121ffbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:25:22.799756 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6b77ce-4d59-4731-8351-9865e121ffbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:25:22.799756 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6b77ce-4d59-4731-8351-9865e121ffbb admin admin] PATH_INFO: `/v3/projects/3dbb1f888fb3409e93187a867375fb82` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:25:22.799756 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6b77ce-4d59-4731-8351-9865e121ffbb admin admin] RBAC: Authorizing `identity:delete_project(project_id=3dbb1f888fb3409e93187a867375fb82)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:25:22.799756 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6b77ce-4d59-4731-8351-9865e121ffbb admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:25:22.857308 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-4d6b77ce-4d59-4731-8351-9865e121ffbb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3dbb1f888fb3409e93187a867375fb82', 'request_id': 'req-4d6b77ce-4d59-4731-8351-9865e121ffbb'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:25:22.863375 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-4d6b77ce-4d59-4731-8351-9865e121ffbb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3dbb1f888fb3409e93187a867375fb82', 'request_id': 'req-4d6b77ce-4d59-4731-8351-9865e121ffbb'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:25:22.869002 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-4d6b77ce-4d59-4731-8351-9865e121ffbb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3dbb1f888fb3409e93187a867375fb82', 'request_id': 'req-4d6b77ce-4d59-4731-8351-9865e121ffbb'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:25:22.883230 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-4d6b77ce-4d59-4731-8351-9865e121ffbb admin admin] The token cache is being invalidate because project 3dbb1f888fb3409e93187a867375fb82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80236) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 16:25:22.883551 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-4d6b77ce-4d59-4731-8351-9865e121ffbb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:25:22.884192 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-4d6b77ce-4d59-4731-8351-9865e121ffbb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:25:22.885781 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1472/2914] 10.222.0.99 () {58 vars in 1232 bytes} [Thu Jul 14 16:25:22 2022] DELETE /identity/v3/projects/3dbb1f888fb3409e93187a867375fb82 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:25:22.890803 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3582f39f-bf23-400a-86d2-e49cb3bdc68b None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:25:22.891093 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3582f39f-bf23-400a-86d2-e49cb3bdc68b None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:25:22.891374 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3582f39f-bf23-400a-86d2-e49cb3bdc68b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:25:22.909490 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-3582f39f-bf23-400a-86d2-e49cb3bdc68b None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:25:22.910705 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-3582f39f-bf23-400a-86d2-e49cb3bdc68b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:25:22.927516 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:25:22.972831 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1443/2915] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 16:25:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:25:22.980873 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64744729-32ca-40bc-8478-04919334ed57 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:25:22.981176 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64744729-32ca-40bc-8478-04919334ed57 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:25:22.981468 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64744729-32ca-40bc-8478-04919334ed57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:25:23.001569 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-64744729-32ca-40bc-8478-04919334ed57 None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:25:23.002808 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-64744729-32ca-40bc-8478-04919334ed57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:25:23.011327 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1473/2916] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 16:25:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:25:23.015698 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3582f39f-bf23-400a-86d2-e49cb3bdc68b None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:25:23.026575 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3582f39f-bf23-400a-86d2-e49cb3bdc68b None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:25:23.032819 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9980c79-c3e9-4998-91d4-908bf0bb8f22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:25:23.033501 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9980c79-c3e9-4998-91d4-908bf0bb8f22 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:25:23.033790 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9980c79-c3e9-4998-91d4-908bf0bb8f22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:25:23.034071 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9980c79-c3e9-4998-91d4-908bf0bb8f22 admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:25:23.034834 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9980c79-c3e9-4998-91d4-908bf0bb8f22 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 16:25:23.035150 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9980c79-c3e9-4998-91d4-908bf0bb8f22 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:25:23.036787 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9980c79-c3e9-4998-91d4-908bf0bb8f22 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:25:23.051354 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1444/2917] 10.222.0.99 () {58 vars in 1152 bytes} [Thu Jul 14 16:25:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 16:25:23.058005 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64744729-32ca-40bc-8478-04919334ed57 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:25:23.075976 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64744729-32ca-40bc-8478-04919334ed57 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:25:23.083608 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e5e66e-d288-4e23-aa55-47a551e772f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:25:23.085372 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e5e66e-d288-4e23-aa55-47a551e772f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:25:23.086123 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e5e66e-d288-4e23-aa55-47a551e772f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:25:23.086668 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e5e66e-d288-4e23-aa55-47a551e772f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:25:23.088161 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e5e66e-d288-4e23-aa55-47a551e772f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:25:23.091781 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e5e66e-d288-4e23-aa55-47a551e772f8 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:25:23.134064 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1474/2918] 10.222.0.99 () {60 vars in 1152 bytes} [Thu Jul 14 16:25:23 2022] POST /identity/v3/projects => generated 385 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 16:25:23.138087 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9980c79-c3e9-4998-91d4-908bf0bb8f22 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:25:23.148690 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9980c79-c3e9-4998-91d4-908bf0bb8f22 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:25:23.204251 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a2f601-9893-4fcd-87d6-e548f8b5d968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:25:23.204935 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a2f601-9893-4fcd-87d6-e548f8b5d968 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:25:23.205189 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a2f601-9893-4fcd-87d6-e548f8b5d968 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:25:23.205466 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a2f601-9893-4fcd-87d6-e548f8b5d968 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:25:23.206163 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a2f601-9893-4fcd-87d6-e548f8b5d968 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:25:23.208110 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a2f601-9893-4fcd-87d6-e548f8b5d968 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:25:23.227782 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1445/2919] 10.222.0.99 () {60 vars in 1143 bytes} [Thu Jul 14 16:25:23 2022] POST /identity/v3/users => generated 293 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 16:25:23.231954 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e5e66e-d288-4e23-aa55-47a551e772f8 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:25:23.243145 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e5e66e-d288-4e23-aa55-47a551e772f8 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:25:23.247763 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e753ccd2-ccf0-42fd-8de9-e4b2f95f93cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:25:23.248425 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e753ccd2-ccf0-42fd-8de9-e4b2f95f93cc admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:25:23.248738 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e753ccd2-ccf0-42fd-8de9-e4b2f95f93cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:25:23.249017 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e753ccd2-ccf0-42fd-8de9-e4b2f95f93cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:25:23.249747 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e753ccd2-ccf0-42fd-8de9-e4b2f95f93cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:25:23.251349 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e753ccd2-ccf0-42fd-8de9-e4b2f95f93cc admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:25:23.262375 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1475/2920] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 16:25:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:25:23.266335 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a2f601-9893-4fcd-87d6-e548f8b5d968 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:25:23.276460 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a2f601-9893-4fcd-87d6-e548f8b5d968 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:25:23.281058 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c30495-9b32-4afa-bd0a-3aed7f116af9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:25:23.281715 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c30495-9b32-4afa-bd0a-3aed7f116af9 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:25:23.282000 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c30495-9b32-4afa-bd0a-3aed7f116af9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:25:23.282279 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c30495-9b32-4afa-bd0a-3aed7f116af9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:25:23.283001 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c30495-9b32-4afa-bd0a-3aed7f116af9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:25:23.284617 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c30495-9b32-4afa-bd0a-3aed7f116af9 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:25:23.296289 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1446/2921] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 16:25:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:25:23.300501 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e753ccd2-ccf0-42fd-8de9-e4b2f95f93cc admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:25:23.311542 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e753ccd2-ccf0-42fd-8de9-e4b2f95f93cc admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:25:23.316258 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840b4c6d-4ddb-4f64-9fa4-cc71c0f9d961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:25:23.317150 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840b4c6d-4ddb-4f64-9fa4-cc71c0f9d961 admin admin] REQUEST_METHOD: `PUT` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:25:23.317437 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840b4c6d-4ddb-4f64-9fa4-cc71c0f9d961 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:25:23.317717 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840b4c6d-4ddb-4f64-9fa4-cc71c0f9d961 admin admin] PATH_INFO: `/v3/projects/a4f95de73a8f4a0ba3d4fdaab174a1c2/users/755f5db986a24ac9a7bd5c77a257df60/roles/b8c98fddc0fd4436b577fd1d3361bc38` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:25:23.334002 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840b4c6d-4ddb-4f64-9fa4-cc71c0f9d961 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4f95de73a8f4a0ba3d4fdaab174a1c2, user_id=755f5db986a24ac9a7bd5c77a257df60, role_id=b8c98fddc0fd4436b577fd1d3361bc38)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:25:23.336852 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840b4c6d-4ddb-4f64-9fa4-cc71c0f9d961 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:25:23.349972 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1476/2922] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 16:25:23 2022] PUT /identity/v3/projects/a4f95de73a8f4a0ba3d4fdaab174a1c2/users/755f5db986a24ac9a7bd5c77a257df60/roles/b8c98fddc0fd4436b577fd1d3361bc38 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:25:23.355368 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950c495f-3fc9-45f1-8c40-788ce27d9d2b None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:25:23.355689 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950c495f-3fc9-45f1-8c40-788ce27d9d2b None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:25:23.355980 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950c495f-3fc9-45f1-8c40-788ce27d9d2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:25:23.374513 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-950c495f-3fc9-45f1-8c40-788ce27d9d2b None None] MFA Rules not processed for user `3e33ee1d9fa74c5fa2f58e221ad2d0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:25:23.375847 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-950c495f-3fc9-45f1-8c40-788ce27d9d2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:25:23.436798 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1447/2923] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 16:25:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:25:23.447852 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840b4c6d-4ddb-4f64-9fa4-cc71c0f9d961 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:25:23.449541 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-840b4c6d-4ddb-4f64-9fa4-cc71c0f9d961 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:25:23.473058 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:25:23.511977 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840b4c6d-4ddb-4f64-9fa4-cc71c0f9d961 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:25:23.515640 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ec1224-aa92-4569-80d1-0486ce5ebd58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:25:23.516291 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ec1224-aa92-4569-80d1-0486ce5ebd58 service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:25:23.516581 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ec1224-aa92-4569-80d1-0486ce5ebd58 service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:25:23.516865 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ec1224-aa92-4569-80d1-0486ce5ebd58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:25:23.527395 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ec1224-aa92-4569-80d1-0486ce5ebd58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:25:23.528880 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ec1224-aa92-4569-80d1-0486ce5ebd58 service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:25:23.539273 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1477/2924] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:25:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:25:27.989816 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320a17d1-c39a-4c97-9534-8272228da7a7 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:25:27.990330 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320a17d1-c39a-4c97-9534-8272228da7a7 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:25:27.990437 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320a17d1-c39a-4c97-9534-8272228da7a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:25:28.018238 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-320a17d1-c39a-4c97-9534-8272228da7a7 None None] MFA Rules not processed for user `755f5db986a24ac9a7bd5c77a257df60`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:25:28.019698 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-320a17d1-c39a-4c97-9534-8272228da7a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:25:28.040536 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:25:28.100583 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1448/2925] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 16:25:27 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:25:28.105569 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ec1224-aa92-4569-80d1-0486ce5ebd58 service neutron] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:25:28.115820 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ec1224-aa92-4569-80d1-0486ce5ebd58 service neutron] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:25:28.121307 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe5f95c-144f-4fd2-8de3-a25a31271be6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:25:28.121979 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe5f95c-144f-4fd2-8de3-a25a31271be6 admin admin] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:25:28.122264 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe5f95c-144f-4fd2-8de3-a25a31271be6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:25:28.122541 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe5f95c-144f-4fd2-8de3-a25a31271be6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:25:28.123272 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe5f95c-144f-4fd2-8de3-a25a31271be6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:25:28.125223 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe5f95c-144f-4fd2-8de3-a25a31271be6 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:25:28.155097 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1478/2926] 10.222.0.99 () {60 vars in 1152 bytes} [Thu Jul 14 16:25:28 2022] POST /identity/v3/projects => generated 385 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 16:25:28.159004 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320a17d1-c39a-4c97-9534-8272228da7a7 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:25:28.170479 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320a17d1-c39a-4c97-9534-8272228da7a7 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:25:28.229833 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726a0230-f41e-4887-b797-c2c990cb7dbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:25:28.230432 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726a0230-f41e-4887-b797-c2c990cb7dbb admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:25:28.230720 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726a0230-f41e-4887-b797-c2c990cb7dbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:25:28.231003 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726a0230-f41e-4887-b797-c2c990cb7dbb admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:25:28.231756 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726a0230-f41e-4887-b797-c2c990cb7dbb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:25:28.233724 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726a0230-f41e-4887-b797-c2c990cb7dbb admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:25:28.253900 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1449/2927] 10.222.0.99 () {60 vars in 1143 bytes} [Thu Jul 14 16:25:28 2022] POST /identity/v3/users => generated 299 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 16:25:28.258633 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe5f95c-144f-4fd2-8de3-a25a31271be6 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:25:28.268993 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe5f95c-144f-4fd2-8de3-a25a31271be6 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:25:28.274355 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0262d8a3-88b2-46db-8cce-5a1d82b410a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:25:28.275026 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0262d8a3-88b2-46db-8cce-5a1d82b410a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:25:28.275317 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0262d8a3-88b2-46db-8cce-5a1d82b410a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:25:28.275622 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0262d8a3-88b2-46db-8cce-5a1d82b410a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:25:28.276364 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0262d8a3-88b2-46db-8cce-5a1d82b410a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:25:28.277967 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0262d8a3-88b2-46db-8cce-5a1d82b410a2 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:25:28.289820 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1479/2928] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 16:25:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 16:25:28.294193 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726a0230-f41e-4887-b797-c2c990cb7dbb admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:25:28.305292 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726a0230-f41e-4887-b797-c2c990cb7dbb admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:25:28.310327 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7983bafa-437d-4f90-a8cd-83588996e80f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:25:28.311234 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7983bafa-437d-4f90-a8cd-83588996e80f admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:25:28.311522 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7983bafa-437d-4f90-a8cd-83588996e80f admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:25:28.311834 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7983bafa-437d-4f90-a8cd-83588996e80f admin admin] PATH_INFO: `/v3/projects/e85ee652de05440c81ae572a54109a99/users/5eeb37e0c4fe49aa96b0fd96bb047211/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:25:28.328542 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7983bafa-437d-4f90-a8cd-83588996e80f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e85ee652de05440c81ae572a54109a99, user_id=5eeb37e0c4fe49aa96b0fd96bb047211, role_id=d0f6ba1314f14d9f9c816bdb5d0c3aac)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:25:28.331191 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7983bafa-437d-4f90-a8cd-83588996e80f admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:25:28.346900 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1450/2929] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 16:25:28 2022] PUT /identity/v3/projects/e85ee652de05440c81ae572a54109a99/users/5eeb37e0c4fe49aa96b0fd96bb047211/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:25:28.350923 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0262d8a3-88b2-46db-8cce-5a1d82b410a2 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:25:28.361140 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0262d8a3-88b2-46db-8cce-5a1d82b410a2 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:25:28.419576 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e65c12e-b9eb-489d-b572-80770864ac6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:25:28.420281 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e65c12e-b9eb-489d-b572-80770864ac6a admin admin] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:25:28.420572 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e65c12e-b9eb-489d-b572-80770864ac6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:25:28.420854 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e65c12e-b9eb-489d-b572-80770864ac6a admin admin] PATH_INFO: `/v3/roles` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:25:28.421590 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e65c12e-b9eb-489d-b572-80770864ac6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:25:28.423230 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e65c12e-b9eb-489d-b572-80770864ac6a admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:25:28.433950 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1480/2930] 10.222.0.99 () {58 vars in 1121 bytes} [Thu Jul 14 16:25:28 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:25:28.438703 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7983bafa-437d-4f90-a8cd-83588996e80f admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:25:28.451188 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7983bafa-437d-4f90-a8cd-83588996e80f admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:25:28.455846 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d96f31e-a7c6-49db-9723-2a5c812dcb81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:25:28.456785 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d96f31e-a7c6-49db-9723-2a5c812dcb81 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:25:28.457110 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d96f31e-a7c6-49db-9723-2a5c812dcb81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:25:28.457447 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d96f31e-a7c6-49db-9723-2a5c812dcb81 admin admin] PATH_INFO: `/v3/projects/e85ee652de05440c81ae572a54109a99/users/5eeb37e0c4fe49aa96b0fd96bb047211/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:25:28.461056 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d96f31e-a7c6-49db-9723-2a5c812dcb81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e85ee652de05440c81ae572a54109a99, user_id=5eeb37e0c4fe49aa96b0fd96bb047211, role_id=d0f6ba1314f14d9f9c816bdb5d0c3aac)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:25:28.465195 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d96f31e-a7c6-49db-9723-2a5c812dcb81 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:25:28.475735 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1451/2931] 10.222.0.99 () {60 vars in 1482 bytes} [Thu Jul 14 16:25:28 2022] PUT /identity/v3/projects/e85ee652de05440c81ae572a54109a99/users/5eeb37e0c4fe49aa96b0fd96bb047211/roles/d0f6ba1314f14d9f9c816bdb5d0c3aac => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:25:33.050701 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71539ef7-9790-4d1c-9dfa-6b98bc00655f None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:25:33.051972 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71539ef7-9790-4d1c-9dfa-6b98bc00655f None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:25:33.052657 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71539ef7-9790-4d1c-9dfa-6b98bc00655f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:25:33.124760 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-71539ef7-9790-4d1c-9dfa-6b98bc00655f None None] MFA Rules not processed for user `5eeb37e0c4fe49aa96b0fd96bb047211`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:25:33.127044 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-71539ef7-9790-4d1c-9dfa-6b98bc00655f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:25:33.244307 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1481/2932] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 16:25:33 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:25:33.258504 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d96f31e-a7c6-49db-9723-2a5c812dcb81 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:25:33.260971 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-2d96f31e-a7c6-49db-9723-2a5c812dcb81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:25:33.352513 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d96f31e-a7c6-49db-9723-2a5c812dcb81 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:25:33.357258 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230edc93-8b8d-45f6-8fd5-3b503b33b6ed service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605e230d50ce44629f554c95b2f79ead', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:25:33.358029 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230edc93-8b8d-45f6-8fd5-3b503b33b6ed service glance] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:25:33.358390 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230edc93-8b8d-45f6-8fd5-3b503b33b6ed service glance] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:25:33.358845 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230edc93-8b8d-45f6-8fd5-3b503b33b6ed service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:25:33.396584 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:25:33.443221 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230edc93-8b8d-45f6-8fd5-3b503b33b6ed service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:25:33.445509 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230edc93-8b8d-45f6-8fd5-3b503b33b6ed service glance] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:25:33.458590 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1452/2933] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:25:33 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:25:33.576104 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbcbd41-9316-47b6-bc1c-06e4c7e6bbee None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:25:33.576508 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbcbd41-9316-47b6-bc1c-06e4c7e6bbee None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:25:33.576868 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbcbd41-9316-47b6-bc1c-06e4c7e6bbee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:25:33.596290 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:25:33.654983 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-4dbcbd41-9316-47b6-bc1c-06e4c7e6bbee None None] MFA Rules not processed for user `adaeadf613954d7393b3fd3bae68c362`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:25:33.656294 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-4dbcbd41-9316-47b6-bc1c-06e4c7e6bbee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:25:33.729659 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1482/2934] 10.222.0.99 () {58 vars in 1009 bytes} [Thu Jul 14 16:25:33 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:25:33.780509 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230edc93-8b8d-45f6-8fd5-3b503b33b6ed service glance] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:25:33.781956 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-230edc93-8b8d-45f6-8fd5-3b503b33b6ed service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:25:33.878054 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230edc93-8b8d-45f6-8fd5-3b503b33b6ed service glance] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:25:33.884204 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86574548-f33a-49ad-8f90-b60bbb978dd4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd2ec3b3ebc4aacb322c95270374451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:25:33.884850 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86574548-f33a-49ad-8f90-b60bbb978dd4 service swift] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:25:33.885137 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86574548-f33a-49ad-8f90-b60bbb978dd4 service swift] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:25:33.885437 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86574548-f33a-49ad-8f90-b60bbb978dd4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:25:33.894273 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86574548-f33a-49ad-8f90-b60bbb978dd4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:25:33.896053 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86574548-f33a-49ad-8f90-b60bbb978dd4 service swift] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:25:33.905308 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1453/2935] 10.222.0.99 () {60 vars in 1384 bytes} [Thu Jul 14 16:25:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 16:25:42.699746 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d9fcaf-b1e8-47c9-95e6-c84507137866 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:25:42.700268 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d9fcaf-b1e8-47c9-95e6-c84507137866 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:25:42.700363 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d9fcaf-b1e8-47c9-95e6-c84507137866 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:25:42.718260 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-38d9fcaf-b1e8-47c9-95e6-c84507137866 None None] MFA Rules not processed for user `755f5db986a24ac9a7bd5c77a257df60`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:25:42.719475 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-38d9fcaf-b1e8-47c9-95e6-c84507137866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:25:42.727309 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1483/2936] 10.222.0.99 () {58 vars in 957 bytes} [Thu Jul 14 16:25:42 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:25:42.766022 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86574548-f33a-49ad-8f90-b60bbb978dd4 service swift] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:25:42.776103 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86574548-f33a-49ad-8f90-b60bbb978dd4 service swift] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:25:42.779538 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6668a65e-c3e3-46e4-a53c-566d99b1897b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605e230d50ce44629f554c95b2f79ead', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:25:42.780196 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6668a65e-c3e3-46e4-a53c-566d99b1897b service glance] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:25:42.780480 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6668a65e-c3e3-46e4-a53c-566d99b1897b service glance] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:25:42.780763 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6668a65e-c3e3-46e4-a53c-566d99b1897b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:25:42.782493 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-6668a65e-c3e3-46e4-a53c-566d99b1897b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:25:42.854922 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6668a65e-c3e3-46e4-a53c-566d99b1897b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:25:42.856402 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6668a65e-c3e3-46e4-a53c-566d99b1897b service glance] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:25:42.867442 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1454/2937] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:25:42 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 16:25:43.097341 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80b87e1-da13-42e8-a0f8-42d40cf5c68a None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:25:43.097633 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80b87e1-da13-42e8-a0f8-42d40cf5c68a None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:25:43.097916 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80b87e1-da13-42e8-a0f8-42d40cf5c68a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:25:43.134902 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-e80b87e1-da13-42e8-a0f8-42d40cf5c68a None None] MFA Rules not processed for user `0d0261759f9743b1a13a760ca9159305`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:25:43.136159 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-e80b87e1-da13-42e8-a0f8-42d40cf5c68a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:25:43.187867 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1484/2938] 10.222.0.99 () {58 vars in 1061 bytes} [Thu Jul 14 16:25:43 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:25:43.194464 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6668a65e-c3e3-46e4-a53c-566d99b1897b service glance] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:25:43.203420 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6668a65e-c3e3-46e4-a53c-566d99b1897b service glance] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:25:43.207096 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83cddf5-e2df-44a9-9f45-4833378c730a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:25:43.207788 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83cddf5-e2df-44a9-9f45-4833378c730a service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:25:43.208056 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83cddf5-e2df-44a9-9f45-4833378c730a service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:25:43.209067 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83cddf5-e2df-44a9-9f45-4833378c730a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:25:43.211097 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c83cddf5-e2df-44a9-9f45-4833378c730a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:25:43.221066 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83cddf5-e2df-44a9-9f45-4833378c730a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:25:43.222497 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83cddf5-e2df-44a9-9f45-4833378c730a service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:25:43.230705 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1455/2939] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:25:43 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 16:25:43.571435 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80b87e1-da13-42e8-a0f8-42d40cf5c68a None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:25:43.573308 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-e80b87e1-da13-42e8-a0f8-42d40cf5c68a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:25:43.584025 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80b87e1-da13-42e8-a0f8-42d40cf5c68a None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:25:43.587636 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1233596-7154-42b5-8efc-64fa1c568d6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:25:43.588227 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1233596-7154-42b5-8efc-64fa1c568d6c service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:25:43.588511 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1233596-7154-42b5-8efc-64fa1c568d6c service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:25:43.588787 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1233596-7154-42b5-8efc-64fa1c568d6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:25:43.590513 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-f1233596-7154-42b5-8efc-64fa1c568d6c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:25:43.604573 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1233596-7154-42b5-8efc-64fa1c568d6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:25:43.606083 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1233596-7154-42b5-8efc-64fa1c568d6c service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:25:43.618635 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1485/2940] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:25:43 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:25:44.577832 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83cddf5-e2df-44a9-9f45-4833378c730a service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:25:44.579358 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c83cddf5-e2df-44a9-9f45-4833378c730a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:25:44.590076 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:25:44.663898 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83cddf5-e2df-44a9-9f45-4833378c730a service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:25:44.669080 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92172c3b-3f82-4ad1-9012-e84db71c9ba4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc157c485f794e968a60d8405c9310ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:25:44.669737 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92172c3b-3f82-4ad1-9012-e84db71c9ba4 service placement] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:25:44.670081 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92172c3b-3f82-4ad1-9012-e84db71c9ba4 service placement] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:25:44.670361 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92172c3b-3f82-4ad1-9012-e84db71c9ba4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:25:44.672273 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-92172c3b-3f82-4ad1-9012-e84db71c9ba4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:25:44.689897 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92172c3b-3f82-4ad1-9012-e84db71c9ba4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:25:44.691702 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92172c3b-3f82-4ad1-9012-e84db71c9ba4 service placement] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:25:44.704418 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1456/2941] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:25:44 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:25:49.072287 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3927b72-8890-47d1-b6a0-5097f2e462a9 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:25:49.072814 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3927b72-8890-47d1-b6a0-5097f2e462a9 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:25:49.072901 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3927b72-8890-47d1-b6a0-5097f2e462a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:25:49.094205 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-c3927b72-8890-47d1-b6a0-5097f2e462a9 None None] MFA Rules not processed for user `adaeadf613954d7393b3fd3bae68c362`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:25:49.095344 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-c3927b72-8890-47d1-b6a0-5097f2e462a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:25:49.104018 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1486/2942] 10.222.0.99 () {58 vars in 1009 bytes} [Thu Jul 14 16:25:49 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:25:49.116309 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92172c3b-3f82-4ad1-9012-e84db71c9ba4 service placement] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:25:49.128194 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92172c3b-3f82-4ad1-9012-e84db71c9ba4 service placement] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:25:49.134680 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a02dc42-222d-45c0-b2fc-38ac5904856d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd2ec3b3ebc4aacb322c95270374451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:25:49.134913 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a02dc42-222d-45c0-b2fc-38ac5904856d service swift] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:25:49.135269 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a02dc42-222d-45c0-b2fc-38ac5904856d service swift] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:25:49.136043 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a02dc42-222d-45c0-b2fc-38ac5904856d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:25:49.146207 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a02dc42-222d-45c0-b2fc-38ac5904856d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:25:49.148227 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a02dc42-222d-45c0-b2fc-38ac5904856d service swift] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:25:49.159398 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1457/2943] 10.222.0.99 () {60 vars in 1384 bytes} [Thu Jul 14 16:25:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 16:25:58.220931 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3927b72-8890-47d1-b6a0-5097f2e462a9 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:25:58.231006 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3927b72-8890-47d1-b6a0-5097f2e462a9 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:25:58.234740 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8cf432-c395-41a8-9315-1bfa7ea9d1be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:25:58.235407 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8cf432-c395-41a8-9315-1bfa7ea9d1be service neutron] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:25:58.235755 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8cf432-c395-41a8-9315-1bfa7ea9d1be service neutron] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:25:58.236032 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8cf432-c395-41a8-9315-1bfa7ea9d1be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:25:58.237807 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-1a8cf432-c395-41a8-9315-1bfa7ea9d1be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:25:58.248027 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8cf432-c395-41a8-9315-1bfa7ea9d1be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:25:58.249495 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8cf432-c395-41a8-9315-1bfa7ea9d1be service neutron] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:25:58.259631 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1487/2944] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:25:58 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:25:58.995414 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a02dc42-222d-45c0-b2fc-38ac5904856d service swift] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:25:58.999243 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-9a02dc42-222d-45c0-b2fc-38ac5904856d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:25:59.020632 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a02dc42-222d-45c0-b2fc-38ac5904856d service swift] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:25:59.030295 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042d73c3-bdaf-4b9e-a293-7c9378ad892e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9b1c2622934c5185f3b5f1ea623bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:25:59.031094 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042d73c3-bdaf-4b9e-a293-7c9378ad892e service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:25:59.032739 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042d73c3-bdaf-4b9e-a293-7c9378ad892e service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:25:59.033069 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042d73c3-bdaf-4b9e-a293-7c9378ad892e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:25:59.035360 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-042d73c3-bdaf-4b9e-a293-7c9378ad892e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:25:59.048771 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042d73c3-bdaf-4b9e-a293-7c9378ad892e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:25:59.050441 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042d73c3-bdaf-4b9e-a293-7c9378ad892e service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:25:59.064659 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1458/2945] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:25:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:26:03.774999 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8cf432-c395-41a8-9315-1bfa7ea9d1be service neutron] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:26:03.777117 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-1a8cf432-c395-41a8-9315-1bfa7ea9d1be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:26:03.789399 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=87 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:26:03.797438 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8cf432-c395-41a8-9315-1bfa7ea9d1be service neutron] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:26:03.804750 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f61b020-4b71-4196-935d-2d7170fd66df service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9b1c2622934c5185f3b5f1ea623bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:26:03.806125 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f61b020-4b71-4196-935d-2d7170fd66df service nova] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:26:03.806421 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f61b020-4b71-4196-935d-2d7170fd66df service nova] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:26:03.806711 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f61b020-4b71-4196-935d-2d7170fd66df service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:26:03.808611 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-4f61b020-4b71-4196-935d-2d7170fd66df service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:26:03.822185 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f61b020-4b71-4196-935d-2d7170fd66df service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:26:03.823818 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f61b020-4b71-4196-935d-2d7170fd66df service nova] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:26:03.835254 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1488/2946] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:26:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:26:09.059038 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=41 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:26:13.830151 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=5 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:26:16.942280 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042d73c3-bdaf-4b9e-a293-7c9378ad892e service nova] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:26:16.951379 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042d73c3-bdaf-4b9e-a293-7c9378ad892e service nova] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:26:16.955082 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e47a1fe-1118-4435-be32-272067d90d99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d0261759f9743b1a13a760ca9159305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:26:16.955797 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e47a1fe-1118-4435-be32-272067d90d99 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:26:16.956513 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e47a1fe-1118-4435-be32-272067d90d99 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:26:16.956513 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e47a1fe-1118-4435-be32-272067d90d99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:26:16.965368 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e47a1fe-1118-4435-be32-272067d90d99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:26:16.966797 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e47a1fe-1118-4435-be32-272067d90d99 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:26:16.976933 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1459/2947] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:26:16 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:26:26.972793 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=6 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:27:33.390388 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f61b020-4b71-4196-935d-2d7170fd66df service nova] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:27:33.423693 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f61b020-4b71-4196-935d-2d7170fd66df service nova] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:27:33.430198 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e733e39b-9476-4989-b5e9-25183f8584c8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9b1c2622934c5185f3b5f1ea623bc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:27:33.430934 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e733e39b-9476-4989-b5e9-25183f8584c8 service nova] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:27:33.431297 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e733e39b-9476-4989-b5e9-25183f8584c8 service nova] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:27:33.431679 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e733e39b-9476-4989-b5e9-25183f8584c8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:27:33.433893 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-e733e39b-9476-4989-b5e9-25183f8584c8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:27:33.460282 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e733e39b-9476-4989-b5e9-25183f8584c8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:27:33.462037 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e733e39b-9476-4989-b5e9-25183f8584c8 service nova] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:27:33.476992 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1489/2948] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:27:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 16:27:39.667422 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfecc066-8cc7-4dd4-9542-fac8a606f94e None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:27:39.668051 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfecc066-8cc7-4dd4-9542-fac8a606f94e None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:27:39.668147 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfecc066-8cc7-4dd4-9542-fac8a606f94e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:27:39.696921 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-bfecc066-8cc7-4dd4-9542-fac8a606f94e None None] MFA Rules not processed for user `adaeadf613954d7393b3fd3bae68c362`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:27:39.698161 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-bfecc066-8cc7-4dd4-9542-fac8a606f94e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:27:39.706269 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1460/2949] 10.222.0.99 () {58 vars in 967 bytes} [Thu Jul 14 16:27:39 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:27:39.746760 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b783f27-4994-4fee-9668-2d074bd84f06 None None] REQUEST_METHOD: `POST` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:27:39.747197 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b783f27-4994-4fee-9668-2d074bd84f06 None None] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:27:39.747330 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b783f27-4994-4fee-9668-2d074bd84f06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:27:39.785357 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.auth.core [None req-8b783f27-4994-4fee-9668-2d074bd84f06 None None] MFA Rules not processed for user `ddd2ec3b3ebc4aacb322c95270374451`. Rule list: `[]` (Enabled: `True`). {{(pid=80236) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:27:39.786606 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-8b783f27-4994-4fee-9668-2d074bd84f06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:27:39.796057 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1490/2950] 10.222.0.99 () {58 vars in 1039 bytes} [Thu Jul 14 16:27:39 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:27:39.802510 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfecc066-8cc7-4dd4-9542-fac8a606f94e None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:27:39.815255 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfecc066-8cc7-4dd4-9542-fac8a606f94e None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:27:39.820332 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658a5a60-e057-4ac9-a3b7-1c8ebeb3583b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd2ec3b3ebc4aacb322c95270374451', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:27:39.820973 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658a5a60-e057-4ac9-a3b7-1c8ebeb3583b service swift] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:27:39.821264 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658a5a60-e057-4ac9-a3b7-1c8ebeb3583b service swift] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:27:39.821550 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658a5a60-e057-4ac9-a3b7-1c8ebeb3583b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:27:39.830496 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658a5a60-e057-4ac9-a3b7-1c8ebeb3583b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:27:39.832049 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658a5a60-e057-4ac9-a3b7-1c8ebeb3583b service swift] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:27:39.840059 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1461/2951] 10.222.0.99 () {60 vars in 1384 bytes} [Thu Jul 14 16:27:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 14 16:27:43.410149 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b783f27-4994-4fee-9668-2d074bd84f06 None None] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:27:43.411737 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-8b783f27-4994-4fee-9668-2d074bd84f06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:27:43.437084 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b783f27-4994-4fee-9668-2d074bd84f06 None None] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:27:43.457307 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357a6d2f-2eec-470e-afb3-c8e4fe1d8cd3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc157c485f794e968a60d8405c9310ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:27:43.457996 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357a6d2f-2eec-470e-afb3-c8e4fe1d8cd3 service placement] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:27:43.458287 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357a6d2f-2eec-470e-afb3-c8e4fe1d8cd3 service placement] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:27:43.458572 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357a6d2f-2eec-470e-afb3-c8e4fe1d8cd3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:27:43.460658 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-357a6d2f-2eec-470e-afb3-c8e4fe1d8cd3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:27:43.474425 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357a6d2f-2eec-470e-afb3-c8e4fe1d8cd3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:27:43.476591 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357a6d2f-2eec-470e-afb3-c8e4fe1d8cd3 service placement] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:27:43.490104 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1491/2952] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:27:43 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:27:46.462621 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658a5a60-e057-4ac9-a3b7-1c8ebeb3583b service swift] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:27:46.475334 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658a5a60-e057-4ac9-a3b7-1c8ebeb3583b service swift] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:27:46.484119 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc75173-9615-4017-a93e-92fae30a4acd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc157c485f794e968a60d8405c9310ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:27:46.484716 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc75173-9615-4017-a93e-92fae30a4acd service placement] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:27:46.485005 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc75173-9615-4017-a93e-92fae30a4acd service placement] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:27:46.485285 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc75173-9615-4017-a93e-92fae30a4acd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:27:46.488112 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-bcc75173-9615-4017-a93e-92fae30a4acd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:27:46.502284 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc75173-9615-4017-a93e-92fae30a4acd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:27:46.503926 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc75173-9615-4017-a93e-92fae30a4acd service placement] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:27:46.515994 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1462/2953] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:27:46 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:27:46.541355 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357a6d2f-2eec-470e-afb3-c8e4fe1d8cd3 service placement] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:27:46.618485 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357a6d2f-2eec-470e-afb3-c8e4fe1d8cd3 service placement] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:27:46.623779 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4030bfda-f52b-42a5-aaf8-2cc44defefce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:27:46.624660 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4030bfda-f52b-42a5-aaf8-2cc44defefce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:27:46.624949 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4030bfda-f52b-42a5-aaf8-2cc44defefce admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:27:46.625230 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4030bfda-f52b-42a5-aaf8-2cc44defefce admin admin] PATH_INFO: `/v3/users/755f5db986a24ac9a7bd5c77a257df60` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:27:46.626425 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4030bfda-f52b-42a5-aaf8-2cc44defefce admin admin] RBAC: Authorizing `identity:delete_user(user_id=755f5db986a24ac9a7bd5c77a257df60)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:27:46.628448 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4030bfda-f52b-42a5-aaf8-2cc44defefce admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:27:46.675408 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-4030bfda-f52b-42a5-aaf8-2cc44defefce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '755f5db986a24ac9a7bd5c77a257df60'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:27:46.682531 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-4030bfda-f52b-42a5-aaf8-2cc44defefce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '755f5db986a24ac9a7bd5c77a257df60'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:27:46.695545 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-4030bfda-f52b-42a5-aaf8-2cc44defefce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '755f5db986a24ac9a7bd5c77a257df60'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:27:46.696235 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-4030bfda-f52b-42a5-aaf8-2cc44defefce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '755f5db986a24ac9a7bd5c77a257df60'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:27:46.696843 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-4030bfda-f52b-42a5-aaf8-2cc44defefce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '755f5db986a24ac9a7bd5c77a257df60'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:27:46.713306 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1492/2954] 10.222.0.99 () {58 vars in 1223 bytes} [Thu Jul 14 16:27:46 2022] DELETE /identity/v3/users/755f5db986a24ac9a7bd5c77a257df60 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:27:46.892375 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc75173-9615-4017-a93e-92fae30a4acd service placement] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:27:46.916137 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc75173-9615-4017-a93e-92fae30a4acd service placement] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:27:46.974021 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45c73ec-fdb5-42cb-88f6-abbe5a22d514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:27:46.974888 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45c73ec-fdb5-42cb-88f6-abbe5a22d514 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:27:46.975187 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45c73ec-fdb5-42cb-88f6-abbe5a22d514 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:27:46.975472 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45c73ec-fdb5-42cb-88f6-abbe5a22d514 admin admin] PATH_INFO: `/v3/projects/a4f95de73a8f4a0ba3d4fdaab174a1c2` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:27:46.976692 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45c73ec-fdb5-42cb-88f6-abbe5a22d514 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4f95de73a8f4a0ba3d4fdaab174a1c2)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:27:46.979237 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45c73ec-fdb5-42cb-88f6-abbe5a22d514 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:27:47.026502 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a45c73ec-fdb5-42cb-88f6-abbe5a22d514 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4f95de73a8f4a0ba3d4fdaab174a1c2', 'request_id': 'req-a45c73ec-fdb5-42cb-88f6-abbe5a22d514'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:27:47.033959 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a45c73ec-fdb5-42cb-88f6-abbe5a22d514 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4f95de73a8f4a0ba3d4fdaab174a1c2', 'request_id': 'req-a45c73ec-fdb5-42cb-88f6-abbe5a22d514'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:27:47.040711 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a45c73ec-fdb5-42cb-88f6-abbe5a22d514 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4f95de73a8f4a0ba3d4fdaab174a1c2', 'request_id': 'req-a45c73ec-fdb5-42cb-88f6-abbe5a22d514'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:27:47.062433 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a45c73ec-fdb5-42cb-88f6-abbe5a22d514 admin admin] The token cache is being invalidate because project a4f95de73a8f4a0ba3d4fdaab174a1c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 16:27:47.062656 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a45c73ec-fdb5-42cb-88f6-abbe5a22d514 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:27:47.063320 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a45c73ec-fdb5-42cb-88f6-abbe5a22d514 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:27:47.064923 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1463/2955] 10.222.0.99 () {58 vars in 1232 bytes} [Thu Jul 14 16:27:46 2022] DELETE /identity/v3/projects/a4f95de73a8f4a0ba3d4fdaab174a1c2 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:27:47.068928 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4030bfda-f52b-42a5-aaf8-2cc44defefce admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:27:47.071017 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-4030bfda-f52b-42a5-aaf8-2cc44defefce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:27:47.084740 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4030bfda-f52b-42a5-aaf8-2cc44defefce admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:27:47.098973 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:27:47.165542 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49817b70-53e2-49bf-b4b2-413ac0e27c64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:27:47.166683 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49817b70-53e2-49bf-b4b2-413ac0e27c64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:27:47.166997 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49817b70-53e2-49bf-b4b2-413ac0e27c64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:27:47.167380 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49817b70-53e2-49bf-b4b2-413ac0e27c64 admin admin] PATH_INFO: `/v3/users/5eeb37e0c4fe49aa96b0fd96bb047211` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:27:47.168576 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49817b70-53e2-49bf-b4b2-413ac0e27c64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5eeb37e0c4fe49aa96b0fd96bb047211)` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:27:47.171324 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49817b70-53e2-49bf-b4b2-413ac0e27c64 admin admin] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:27:47.225728 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-49817b70-53e2-49bf-b4b2-413ac0e27c64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5eeb37e0c4fe49aa96b0fd96bb047211'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:27:47.239992 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-49817b70-53e2-49bf-b4b2-413ac0e27c64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5eeb37e0c4fe49aa96b0fd96bb047211'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:27:47.254657 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-49817b70-53e2-49bf-b4b2-413ac0e27c64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5eeb37e0c4fe49aa96b0fd96bb047211'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:27:47.255425 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-49817b70-53e2-49bf-b4b2-413ac0e27c64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5eeb37e0c4fe49aa96b0fd96bb047211'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:27:47.256106 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.notifications [None req-49817b70-53e2-49bf-b4b2-413ac0e27c64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5eeb37e0c4fe49aa96b0fd96bb047211'} {{(pid=80236) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:27:47.272805 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1493/2956] 10.222.0.99 () {58 vars in 1223 bytes} [Thu Jul 14 16:27:47 2022] DELETE /identity/v3/users/5eeb37e0c4fe49aa96b0fd96bb047211 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 16:27:47.464236 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45c73ec-fdb5-42cb-88f6-abbe5a22d514 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:27:47.465738 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a45c73ec-fdb5-42cb-88f6-abbe5a22d514 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:27:47.480064 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45c73ec-fdb5-42cb-88f6-abbe5a22d514 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:27:47.539134 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:27:47.546144 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b23ef5-9ef9-4364-b4c2-1f096315d2ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e33ee1d9fa74c5fa2f58e221ad2d0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae21968b232140b899e10f7d7d033a88', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:27:47.547492 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b23ef5-9ef9-4364-b4c2-1f096315d2ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:27:47.547990 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b23ef5-9ef9-4364-b4c2-1f096315d2ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:27:47.548287 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b23ef5-9ef9-4364-b4c2-1f096315d2ca admin admin] PATH_INFO: `/v3/projects/e85ee652de05440c81ae572a54109a99` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:27:47.550114 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b23ef5-9ef9-4364-b4c2-1f096315d2ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=e85ee652de05440c81ae572a54109a99)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:27:47.552597 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b23ef5-9ef9-4364-b4c2-1f096315d2ca admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:27:47.621410 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-60b23ef5-9ef9-4364-b4c2-1f096315d2ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e85ee652de05440c81ae572a54109a99', 'request_id': 'req-60b23ef5-9ef9-4364-b4c2-1f096315d2ca'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:27:47.630024 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-60b23ef5-9ef9-4364-b4c2-1f096315d2ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e85ee652de05440c81ae572a54109a99', 'request_id': 'req-60b23ef5-9ef9-4364-b4c2-1f096315d2ca'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:27:47.636495 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-60b23ef5-9ef9-4364-b4c2-1f096315d2ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e85ee652de05440c81ae572a54109a99', 'request_id': 'req-60b23ef5-9ef9-4364-b4c2-1f096315d2ca'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:27:47.652653 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-60b23ef5-9ef9-4364-b4c2-1f096315d2ca admin admin] The token cache is being invalidate because project e85ee652de05440c81ae572a54109a99 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 16:27:47.652929 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-60b23ef5-9ef9-4364-b4c2-1f096315d2ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:27:47.654371 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-60b23ef5-9ef9-4364-b4c2-1f096315d2ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:27:47.657050 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1464/2957] 10.222.0.99 () {58 vars in 1232 bytes} [Thu Jul 14 16:27:47 2022] DELETE /identity/v3/projects/e85ee652de05440c81ae572a54109a99 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:27:57.268179 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=48,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:27:57.648944 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=25,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:30:19.588586 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49817b70-53e2-49bf-b4b2-413ac0e27c64 admin admin] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:30:19.589989 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-49817b70-53e2-49bf-b4b2-413ac0e27c64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:30:19.682003 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49817b70-53e2-49bf-b4b2-413ac0e27c64 admin admin] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:30:19.745943 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87f664b-34d0-4db5-91aa-a5194a96e1fd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc157c485f794e968a60d8405c9310ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:30:19.746614 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87f664b-34d0-4db5-91aa-a5194a96e1fd service placement] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:30:19.746904 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87f664b-34d0-4db5-91aa-a5194a96e1fd service placement] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:30:19.747183 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87f664b-34d0-4db5-91aa-a5194a96e1fd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:30:19.749067 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-f87f664b-34d0-4db5-91aa-a5194a96e1fd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:30:19.762182 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87f664b-34d0-4db5-91aa-a5194a96e1fd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:30:19.763805 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87f664b-34d0-4db5-91aa-a5194a96e1fd service placement] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:30:19.784073 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1494/2958] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:30:19 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:30:29.770020 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG dbcounter [-] [80236] Writing DB stats keystone:SELECT=66 {{(pid=80236) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:33:20.564302 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6407bcc1-d209-4103-83a3-79f431b90239 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:33:20.564916 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6407bcc1-d209-4103-83a3-79f431b90239 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:33:20.564916 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6407bcc1-d209-4103-83a3-79f431b90239 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:33:20.600773 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-6407bcc1-d209-4103-83a3-79f431b90239 None None] MFA Rules not processed for user `dc157c485f794e968a60d8405c9310ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:33:20.602010 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-6407bcc1-d209-4103-83a3-79f431b90239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:33:20.611161 dv-cinder-826510-8-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1465/2959] 10.222.0.124 () {58 vars in 1016 bytes} [Thu Jul 14 16:33:20 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:33:20.627434 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87f664b-34d0-4db5-91aa-a5194a96e1fd service placement] Authenticating user token {{(pid=80236) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:33:20.628847 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.fernet_utils [None req-f87f664b-34d0-4db5-91aa-a5194a96e1fd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80236) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:33:20.642162 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87f664b-34d0-4db5-91aa-a5194a96e1fd service placement] Validating token access rules against request {{(pid=80236) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:33:20.647154 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0411e76-01cd-416f-b92c-a2f05d46d6d4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc157c485f794e968a60d8405c9310ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9048d1a6f00443ec92f11471babe5a56', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80236) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:33:20.647823 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0411e76-01cd-416f-b92c-a2f05d46d6d4 service placement] REQUEST_METHOD: `GET` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:33:20.648116 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0411e76-01cd-416f-b92c-a2f05d46d6d4 service placement] SCRIPT_NAME: `/identity` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:33:20.648398 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0411e76-01cd-416f-b92c-a2f05d46d6d4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80236) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:33:20.658006 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0411e76-01cd-416f-b92c-a2f05d46d6d4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:33:20.659365 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0411e76-01cd-416f-b92c-a2f05d46d6d4 service placement] RBAC: Authorization granted {{(pid=80236) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:33:20.669618 dv-cinder-826510-8-iscsi devstack@keystone.service[80236]: [pid: 80236|app: 0|req: 1495/2960] 10.222.0.99 () {60 vars in 1365 bytes} [Thu Jul 14 16:33:20 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0)